Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-08-10T07:52:29Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:52:29Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260810 07:52:30.006451 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38193
--webserver_interface=127.31.139.126
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 07:52:30.321888 32311 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:30.322570 32311 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:30.323055 32311 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:52:30.356539 32311 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:52:30.356863 32311 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:30.357240 32311 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:52:30.357529 32311 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:52:30.395905 32311 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.139.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:30.397445 32311 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:30.399297 32311 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:30.412487 32318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:30.415926 32321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:30.412961 32319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:31.676193 32320 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1259 milliseconds
I20260810 07:52:31.676314 32311 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:52:31.677781 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:31.680500 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:31.684027 32311 hybrid_clock.cc:648] HybridClock initialized: now 1786348351683982 us; error 96 us; skew 500 ppm
I20260810 07:52:31.685098 32311 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:31.692306 32311 webserver.cc:533] Webserver started at http://127.31.139.126:38367/ using document root <none> and password file <none>
I20260810 07:52:31.693516 32311 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:31.693933 32311 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:31.694625 32311 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:31.699412 32311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/instance:
uuid: "72fde54f23f34224ac756f11924c9531"
format_stamp: "Formatted at 2026-08-10 07:52:31 on dist-test-slave-x5fp"
I20260810 07:52:31.700892 32311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal/instance:
uuid: "72fde54f23f34224ac756f11924c9531"
format_stamp: "Formatted at 2026-08-10 07:52:31 on dist-test-slave-x5fp"
I20260810 07:52:31.711338 32311 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260810 07:52:31.718065 32330 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:31.719282 32311 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 07:52:31.719717 32311 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
uuid: "72fde54f23f34224ac756f11924c9531"
format_stamp: "Formatted at 2026-08-10 07:52:31 on dist-test-slave-x5fp"
I20260810 07:52:31.720073 32311 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:31.786602 32311 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:31.788409 32311 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:31.788968 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:31.870397 32311 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.126:38193
I20260810 07:52:31.870508 32408 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.126:38193 every 8 connection(s)
I20260810 07:52:31.873446 32311 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
I20260810 07:52:31.875589 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 32311
I20260810 07:52:31.876217 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal/instance
I20260810 07:52:31.880627 32409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:31.880623 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.125:46845
--webserver_interface=127.31.139.125
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:52:31.892697 32409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:31.926039 32409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:52:31.929103 32331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
W20260810 07:52:31.933843 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
I20260810 07:52:31.983465 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 1
W20260810 07:52:31.988938 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
I20260810 07:52:32.057463 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 2
W20260810 07:52:32.062282 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
I20260810 07:52:32.153831 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 3
W20260810 07:52:32.158975 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
W20260810 07:52:32.228833 32412 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:32.229450 32412 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:32.229839 32412 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:52:32.263818 32412 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:52:32.264118 32412 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:32.264334 32412 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:52:32.264550 32412 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:52:32.302482 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 4
I20260810 07:52:32.303001 32412 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.125:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.139.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:32.305224 32412 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:32.308068 32412 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:32.308588 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
W20260810 07:52:32.320974 32427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:32.608103 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 5
W20260810 07:52:32.616140 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
I20260810 07:52:33.163894 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 6
W20260810 07:52:33.176239 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:46845: Network error: Client connection negotiation failed: client connection to 127.31.139.125:46845: connect: Connection refused (error 111)
W20260810 07:52:32.322947 32429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:32.327593 32434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:33.618222 32432 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1291 milliseconds
I20260810 07:52:33.618355 32412 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:52:33.619985 32412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:33.623373 32412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:33.624851 32412 hybrid_clock.cc:648] HybridClock initialized: now 1786348353624779 us; error 80 us; skew 500 ppm
I20260810 07:52:33.625954 32412 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:33.632671 32412 webserver.cc:533] Webserver started at http://127.31.139.125:45199/ using document root <none> and password file <none>
I20260810 07:52:33.633780 32412 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:33.634017 32412 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:33.634562 32412 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:33.639562 32412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data/instance:
uuid: "d86331b1fae14fb4b49b99f676a8490a"
format_stamp: "Formatted at 2026-08-10 07:52:33 on dist-test-slave-x5fp"
I20260810 07:52:33.640837 32412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal/instance:
uuid: "d86331b1fae14fb4b49b99f676a8490a"
format_stamp: "Formatted at 2026-08-10 07:52:33 on dist-test-slave-x5fp"
I20260810 07:52:33.649116 32412 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 07:52:33.655468 32449 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:33.657063 32412 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20260810 07:52:33.657488 32412 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal
uuid: "d86331b1fae14fb4b49b99f676a8490a"
format_stamp: "Formatted at 2026-08-10 07:52:33 on dist-test-slave-x5fp"
I20260810 07:52:33.658000 32412 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:33.736119 32412 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:33.737949 32412 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:33.738461 32412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:33.815728 32412 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.125:46845
I20260810 07:52:33.815800 32522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.125:46845 every 8 connection(s)
I20260810 07:52:33.819386 32412 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
I20260810 07:52:33.826093 32523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:33.826586 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 32412
I20260810 07:52:33.827029 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-1/wal/instance
I20260810 07:52:33.829383 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:36273
--webserver_interface=127.31.139.124
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:52:33.837489 32523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:33.866940 32523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:33.887039 32523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:52:33.890514 32455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
W20260810 07:52:33.895969 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:33.945856 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 1
W20260810 07:52:33.951289 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.019716 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 2
W20260810 07:52:34.024554 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.116082 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 3
W20260810 07:52:34.121109 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
W20260810 07:52:34.199478 32526 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:34.200460 32526 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:34.201409 32526 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260810 07:52:34.236966 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } attempt: 7
I20260810 07:52:34.250401 32409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:52:34.251127 32526 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:52:34.251536 32526 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:34.251946 32526 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:52:34.252322 32526 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:52:34.257330 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.264642 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 4
W20260810 07:52:34.269309 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.296309 32526 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.139.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:34.297930 32526 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:34.299669 32526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:34.311604 32542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:34.315759 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 1
W20260810 07:52:34.322813 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.404361 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 2
W20260810 07:52:34.413611 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.499385 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 3
W20260810 07:52:34.508389 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.568774 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 5
W20260810 07:52:34.578457 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.649003 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 4
W20260810 07:52:34.659147 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:34.942795 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 5
W20260810 07:52:34.952026 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:35.126092 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 6
W20260810 07:52:35.149222 32523 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
I20260810 07:52:35.504675 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 6
W20260810 07:52:35.530632 32409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:36273: Network error: Client connection negotiation failed: client connection to 127.31.139.124:36273: connect: Connection refused (error 111)
W20260810 07:52:35.715754 32539 debug-util.cc:398] Leaking SignalData structure 0x7b080002ede0 after lost signal to thread 32526
W20260810 07:52:35.910651 32526 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.599s user 0.584s sys 1.009s
W20260810 07:52:35.911130 32526 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.600s user 0.584s sys 1.009s
W20260810 07:52:34.311641 32543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:35.913419 32547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:35.916213 32545 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1599 milliseconds
I20260810 07:52:35.916261 32526 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:52:35.917790 32526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:35.920356 32526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:35.921765 32526 hybrid_clock.cc:648] HybridClock initialized: now 1786348355921662 us; error 107 us; skew 500 ppm
I20260810 07:52:35.922649 32526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:35.929307 32526 webserver.cc:533] Webserver started at http://127.31.139.124:40517/ using document root <none> and password file <none>
I20260810 07:52:35.930366 32526 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:35.930644 32526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:35.931145 32526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:35.935607 32526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data/instance:
uuid: "2c43f834fbcb447587880136f69c7993"
format_stamp: "Formatted at 2026-08-10 07:52:35 on dist-test-slave-x5fp"
I20260810 07:52:35.936841 32526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal/instance:
uuid: "2c43f834fbcb447587880136f69c7993"
format_stamp: "Formatted at 2026-08-10 07:52:35 on dist-test-slave-x5fp"
I20260810 07:52:35.945284 32526 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260810 07:52:35.951349 32562 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:35.952584 32526 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:52:35.953025 32526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal
uuid: "2c43f834fbcb447587880136f69c7993"
format_stamp: "Formatted at 2026-08-10 07:52:35 on dist-test-slave-x5fp"
I20260810 07:52:35.953405 32526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:36.016471 32526 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:36.018110 32526 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:36.018599 32526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:36.095515 32526 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.124:36273
I20260810 07:52:36.095743 32639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.124:36273 every 8 connection(s)
I20260810 07:52:36.098462 32526 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
I20260810 07:52:36.105399 32640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:36.106698 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 32526
I20260810 07:52:36.107259 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-2/wal/instance
I20260810 07:52:36.123423 32640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:36.149610 32640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:36.164055 32640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:52:36.183830 32640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: Bootstrap starting.
I20260810 07:52:36.190940 32640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:36.193472 32640 log.cc:826] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: Log is configured to *not* fsync() on all Append() calls
I20260810 07:52:36.198719 32640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: No bootstrap required, opened a new log
I20260810 07:52:36.209813 32523 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 7
I20260810 07:52:36.224200 32640 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.225461 32640 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:36.225852 32640 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c43f834fbcb447587880136f69c7993, State: Initialized, Role: FOLLOWER
I20260810 07:52:36.227123 32640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.230419 32661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.231298 32661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.231988 32640 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:52:36.233162 32523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Bootstrap starting.
I20260810 07:52:36.241205 32523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:36.243979 32523 log.cc:826] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:52:36.249334 32523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: No bootstrap required, opened a new log
W20260810 07:52:36.269047 32677 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:52:36.269528 32677 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:52:36.274024 32570 tablet.cc:2406] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:52:36.275424 32523 raft_consensus.cc:359] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.276449 32523 raft_consensus.cc:385] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:36.276826 32523 raft_consensus.cc:740] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86331b1fae14fb4b49b99f676a8490a, State: Initialized, Role: FOLLOWER
I20260810 07:52:36.277918 32523 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.281253 32679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.282168 32679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.283074 32523 sys_catalog.cc:565] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:52:36.263334 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: OK, instance_id { permanent_uuid: "72fde54f23f34224ac756f11924c9531" instance_seqno: 1786348351847622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:52:36.299544 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:52:36.334640 32693 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:52:36.335208 32693 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:52:36.325253 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: OK, instance_id { permanent_uuid: "72fde54f23f34224ac756f11924c9531" instance_seqno: 1786348351847622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:52:36.352596 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:36.492566 32661 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:36.493283 32661 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.497881 32661 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:52:36.498283 32486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:52:36.498888 32457 tablet.cc:2406] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:52:36.499383 32486 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 0.
I20260810 07:52:36.501365 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters:
I20260810 07:52:36.504007 32661 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:36.504527 32661 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:36.505052 32661 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:36.513003 32661 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.517388 32486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:52:36.517997 32486 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:36.525527 32486 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 1.
I20260810 07:52:36.526731 32661 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 1 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:52:36.526719 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters:
I20260810 07:52:36.527631 32661 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:36.529644 32661 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 LEADER]: Becoming Leader. State: Replica: 2c43f834fbcb447587880136f69c7993, State: Running, Role: LEADER
I20260810 07:52:36.530697 32661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.538982 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c43f834fbcb447587880136f69c7993. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.539815 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:36.551786 32701 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:52:36.560038 32701 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:52:36.564181 32409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } attempt: 7
I20260810 07:52:36.589383 32486 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Refusing update from remote peer 2c43f834fbcb447587880136f69c7993: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:36.591614 32698 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:36.641443 32409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Bootstrap starting.
I20260810 07:52:36.664675 32679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c43f834fbcb447587880136f69c7993. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.669272 32679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.675413 32706 mvcc.cc:204] Tried to move back new op lower bound from 7316882868570853376 to 7316882868361199616. Current Snapshot: MvccSnapshot[applied={T|T < 7316882868570853376}]
I20260810 07:52:36.680593 32409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:36.683745 32661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.684540 32661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:36.690063 32679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.691278 32679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.699025 32409 log.cc:826] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Log is configured to *not* fsync() on all Append() calls
I20260810 07:52:36.703826 32701 catalog_manager.cc:1383] Generated new cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:52:36.704169 32701 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:52:36.715363 32409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: No bootstrap required, opened a new log
I20260810 07:52:36.740098 32701 catalog_manager.cc:1406] Generated new certificate authority record
I20260810 07:52:36.742287 32701 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:52:36.785849 32701 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: Generated new TSK 0
I20260810 07:52:36.787004 32701 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:52:36.803663 32409 raft_consensus.cc:359] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.804549 32409 raft_consensus.cc:385] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:36.804886 32409 raft_consensus.cc:740] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fde54f23f34224ac756f11924c9531, State: Initialized, Role: FOLLOWER
I20260810 07:52:36.805923 32409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:36.818063 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.65:0
--local_ip_for_outbound_sockets=127.31.139.65
--webserver_interface=127.31.139.65
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 07:52:36.837775 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.838624 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.839944 32409 sys_catalog.cc:565] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:52:36.840160 32372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:36.841074 32372 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 0.
I20260810 07:52:36.842583 32370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:36.850054 32371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:52:36.854736 32370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Refusing update from remote peer 2c43f834fbcb447587880136f69c7993: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:36.859565 32661 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:36.970578 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c43f834fbcb447587880136f69c7993. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.970726 32728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:36.971369 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:36.971421 32728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:37.021620 32734 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:52:37.022027 32734 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: loading cluster ID for follower catalog manager: success
I20260810 07:52:37.030294 32734 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: acquiring CA information for follower catalog manager: success
I20260810 07:52:37.053242 32734 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:52:37.303406 32737 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:37.304196 32737 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:37.307698 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:37.317245 32737 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:37.318699 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:37.320864 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86331b1fae14fb4b49b99f676a8490a; no voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531
I20260810 07:52:37.321768 32737 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:37.341269 32693 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:52:37.341688 32693 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: loading cluster ID for follower catalog manager: success
I20260810 07:52:37.349375 32693 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: acquiring CA information for follower catalog manager: success
I20260810 07:52:37.366437 32693 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:52:37.367146 32661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:37.368039 32661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: LEADER
W20260810 07:52:37.484514 32714 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:37.485380 32714 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:37.486464 32714 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:52:37.556401 32714 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:37.558079 32714 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.65
I20260810 07:52:37.633769 32714 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.139.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.65
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:37.635737 32714 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:37.638113 32714 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:37.673085 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:37.883545 32753 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:37.884145 32753 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:37.896090 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:37.896278 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:37.898211 32753 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:37.898566 32331 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:37.899863 32753 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:38.249578 32754 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:38.257114 32754 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:38.261940 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:38.265440 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:38.267239 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86331b1fae14fb4b49b99f676a8490a; no voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531
I20260810 07:52:38.272634 32754 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:38.273775 32754 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:38.481462 32759 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:38.482095 32759 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:38.485137 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:38.501471 32759 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:38.502938 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:38.505110 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:38.506383 32759 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:39.470122 32761 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:39.480999 32761 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:39.488523 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:39.489006 32761 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:39.491956 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:39.494467 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:39.495571 32761 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:39.754709 32762 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:39.755517 32762 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:39.758461 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:39.759444 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:39.761130 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86331b1fae14fb4b49b99f676a8490a; no voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531
I20260810 07:52:39.767560 32762 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:39.768628 32762 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:52:39.067916 32747 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 32714
W20260810 07:52:39.931984 32747 kernel_stack_watchdog.cc:198] Thread 32714 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 389ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260810 07:52:39.932976 32714 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 2.257s user 0.474s sys 0.860s
W20260810 07:52:37.678288 32750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:39.933389 32714 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 2.257s user 0.474s sys 0.860s
W20260810 07:52:39.950634 32766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:39.950682 32714 server_base.cc:1061] running on GCE node
I20260810 07:52:39.952306 32714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:39.955150 32714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:39.956624 32714 hybrid_clock.cc:648] HybridClock initialized: now 1786348359956596 us; error 73 us; skew 500 ppm
I20260810 07:52:39.958112 32714 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:39.966454 32714 webserver.cc:533] Webserver started at http://127.31.139.65:34237/ using document root <none> and password file <none>
I20260810 07:52:39.967528 32714 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:39.967757 32714 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:39.968288 32714 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:39.973083 32714 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data/instance:
uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
format_stamp: "Formatted at 2026-08-10 07:52:39 on dist-test-slave-x5fp"
I20260810 07:52:39.974246 32714 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/instance:
uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
format_stamp: "Formatted at 2026-08-10 07:52:39 on dist-test-slave-x5fp"
I20260810 07:52:39.982528 32714 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260810 07:52:39.988925 307 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:39.990228 32714 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 07:52:39.990602 32714 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal
uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
format_stamp: "Formatted at 2026-08-10 07:52:39 on dist-test-slave-x5fp"
I20260810 07:52:39.990998 32714 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:40.058955 32714 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:40.060652 32714 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:40.061218 32714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:40.064514 32714 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:52:40.069192 32714 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:52:40.069439 32714 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:40.069736 32714 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:52:40.069924 32714 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:40.252290 32714 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.65:39809
I20260810 07:52:40.252435 497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.65:39809 every 8 connection(s)
I20260810 07:52:40.255141 32714 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
I20260810 07:52:40.265872 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 32714
I20260810 07:52:40.266695 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/instance
I20260810 07:52:40.282462 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.66:0
--local_ip_for_outbound_sockets=127.31.139.66
--webserver_interface=127.31.139.66
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 07:52:40.339476 498 heartbeater.cc:344] Connected to a master server at 127.31.139.124:36273
I20260810 07:52:40.340040 498 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:40.341660 498 heartbeater.cc:507] Master 127.31.139.124:36273 requested a full tablet report, sending...
I20260810 07:52:40.346323 32586 ts_manager.cc:194] Registered new tserver with Master: 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:40.349121 32586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.65:55391
I20260810 07:52:40.354786 516 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:40.355082 500 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:52:40.355427 500 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:40.355496 516 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:40.356170 500 heartbeater.cc:507] Master 127.31.139.126:38193 requested a full tablet report, sending...
I20260810 07:52:40.359416 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:40.359514 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:40.361215 516 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:40.361449 499 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
I20260810 07:52:40.361820 499 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:40.362500 499 heartbeater.cc:507] Master 127.31.139.125:46845 requested a full tablet report, sending...
I20260810 07:52:40.364629 32452 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86331b1fae14fb4b49b99f676a8490a; no voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531
I20260810 07:52:40.365736 32470 ts_manager.cc:194] Registered new tserver with Master: 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:40.365808 516 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:40.363651 32357 ts_manager.cc:194] Registered new tserver with Master: 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:52:40.688829 511 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:40.689391 511 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:40.689975 511 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:52:40.724200 511 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:40.725268 511 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.66
I20260810 07:52:40.765018 511 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.139.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.66
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:40.766467 511 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:40.768249 511 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:40.784566 525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:40.854202 498 heartbeater.cc:499] Master 127.31.139.124:36273 was elected leader, sending a full tablet report...
I20260810 07:52:40.913897 533 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:40.925133 533 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:40.929296 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:40.929916 533 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:40.931303 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:40.934628 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:40.936427 533 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:52:40.949662 534 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:40.950510 534 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:40.954530 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:40.955406 32370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86331b1fae14fb4b49b99f676a8490a in term 1.
I20260810 07:52:40.957644 32452 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a; no voters:
I20260810 07:52:40.963510 534 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:40.964821 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:40.965758 534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:52:40.969610 534 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:40.970923 534 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:40.991721 534 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.002610 32484 raft_consensus.cc:1240] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Rejecting Update request from peer 2c43f834fbcb447587880136f69c7993 for earlier term 1. Current term is 2. Ops: []
I20260810 07:52:41.011445 537 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260810 07:52:41.017707 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:52:41.021782 534 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:41.025382 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993"
I20260810 07:52:41.027244 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86331b1fae14fb4b49b99f676a8490a; no voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531
I20260810 07:52:41.028496 534 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:52:41.031091 536 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 07:52:41.031592 536 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c43f834fbcb447587880136f69c7993, State: Running, Role: LEADER
I20260810 07:52:41.032681 536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.034874 536 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:41.281145 32677 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:52:41.318744 32526 master_runner.cc:233] Existing masters have differing terms: 1 vs 2
I20260810 07:52:41.319443 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:41.381875 32412 master_runner.cc:233] Existing masters have differing terms: 1 vs 2
I20260810 07:52:41.382838 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:41.493871 539 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:41.494679 539 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.499624 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:41.499883 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:41.500634 32601 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fde54f23f34224ac756f11924c9531 in term 2.
I20260810 07:52:41.501235 32484 raft_consensus.cc:2393] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72fde54f23f34224ac756f11924c9531 in current term 2: Already voted for candidate d86331b1fae14fb4b49b99f676a8490a in this term.
I20260810 07:52:41.498026 539 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:41.502909 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531; no voters:
I20260810 07:52:41.504653 539 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:52:41.505482 539 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c43f834fbcb447587880136f69c7993)
I20260810 07:52:41.506147 539 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:41.518779 539 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.523516 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:52:41.523909 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993"
I20260810 07:52:41.524696 32484 raft_consensus.cc:2393] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 72fde54f23f34224ac756f11924c9531 in current term 2: Already voted for candidate d86331b1fae14fb4b49b99f676a8490a in this term.
I20260810 07:52:41.527251 539 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 election: Requested vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:41.534871 32601 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fde54f23f34224ac756f11924c9531 in term 2.
I20260810 07:52:41.536691 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531; no voters: d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:41.537895 539 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:52:41.539463 539 raft_consensus.cc:697] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 LEADER]: Becoming Leader. State: Replica: 72fde54f23f34224ac756f11924c9531, State: Running, Role: LEADER
I20260810 07:52:41.540952 539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.548240 540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72fde54f23f34224ac756f11924c9531. Latest consensus state: current_term: 2 leader_uuid: "72fde54f23f34224ac756f11924c9531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:41.549330 540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:41.552361 545 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:52:41.564877 545 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:52:41.570763 545 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:52:41.571074 545 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:52:41.575584 545 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:52:41.581792 545 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Loaded TSK: 0
I20260810 07:52:41.583774 545 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:52:41.587666 546 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:41.588744 546 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.591382 546 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:41.592592 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:41.593683 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:41.594596 32601 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86331b1fae14fb4b49b99f676a8490a in term 2.
I20260810 07:52:41.596153 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters: 72fde54f23f34224ac756f11924c9531
I20260810 07:52:41.597491 546 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 07:52:41.597945 546 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:41.598379 546 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:52:41.608191 546 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.611416 546 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 3 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:41.612317 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:52:41.613111 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993"
I20260810 07:52:41.613831 32601 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:52:41.623425 32601 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86331b1fae14fb4b49b99f676a8490a in term 3.
I20260810 07:52:41.625057 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters: 72fde54f23f34224ac756f11924c9531
I20260810 07:52:41.626544 546 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:52:41.629338 546 raft_consensus.cc:697] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 LEADER]: Becoming Leader. State: Replica: d86331b1fae14fb4b49b99f676a8490a, State: Running, Role: LEADER
I20260810 07:52:41.630740 546 consensus_queue.cc:237] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.638404 548 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:41.639477 548 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:41.642877 552 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:52:41.654217 552 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:52:41.660152 552 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:52:41.660617 552 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:52:41.666955 552 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:52:41.672438 552 catalog_manager.cc:6103] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Loaded TSK: 0
I20260810 07:52:41.674213 552 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:52:41.926457 32601 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Rejecting Update request from peer 72fde54f23f34224ac756f11924c9531 for earlier term 2. Current term is 3. Ops: []
I20260810 07:52:41.929138 540 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260810 07:52:41.930469 540 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 LEADER]: Stepping down as leader of term 2
I20260810 07:52:41.930925 540 raft_consensus.cc:740] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 72fde54f23f34224ac756f11924c9531, State: Running, Role: LEADER
I20260810 07:52:41.932482 540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:41.934882 540 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:52:42.112840 32601 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Refusing update from remote peer d86331b1fae14fb4b49b99f676a8490a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260810 07:52:42.114621 548 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260810 07:52:42.134241 554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.135334 554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:42.144308 554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.145848 554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:42.148960 548 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.149696 548 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:42.150738 32370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Refusing update from remote peer d86331b1fae14fb4b49b99f676a8490a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260810 07:52:42.150986 32370 pending_rounds.cc:85] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Aborting all ops after (but not including) 4
I20260810 07:52:42.151165 32370 pending_rounds.cc:107] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260810 07:52:42.151346 32370 raft_consensus.cc:2889] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260810 07:52:42.152853 546 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260810 07:52:42.158295 539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.159060 539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:52:42.158808 540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.159463 540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:52:40.785279 524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:42.237749 529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:42.238101 528 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1451 milliseconds
I20260810 07:52:42.238094 511 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:52:42.242393 511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:42.244930 511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:42.246325 511 hybrid_clock.cc:648] HybridClock initialized: now 1786348362246284 us; error 43 us; skew 500 ppm
I20260810 07:52:42.247184 511 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:42.255219 511 webserver.cc:533] Webserver started at http://127.31.139.66:33083/ using document root <none> and password file <none>
I20260810 07:52:42.256218 511 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:42.256484 511 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:42.257109 511 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:42.261619 511 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data/instance:
uuid: "b5bfa873accc428ab48ac19e1207f88d"
format_stamp: "Formatted at 2026-08-10 07:52:42 on dist-test-slave-x5fp"
I20260810 07:52:42.262809 511 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/instance:
uuid: "b5bfa873accc428ab48ac19e1207f88d"
format_stamp: "Formatted at 2026-08-10 07:52:42 on dist-test-slave-x5fp"
I20260810 07:52:42.270957 511 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.001s sys 0.007s
I20260810 07:52:42.276997 572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:42.278167 511 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 07:52:42.278524 511 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal
uuid: "b5bfa873accc428ab48ac19e1207f88d"
format_stamp: "Formatted at 2026-08-10 07:52:42 on dist-test-slave-x5fp"
I20260810 07:52:42.278887 511 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:42.351130 511 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:42.352871 511 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:42.353339 511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:42.356150 511 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:52:42.360540 511 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:52:42.360818 511 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 07:52:42.361090 511 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:52:42.361281 511 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:42.391144 499 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:52:42.391168 500 heartbeater.cc:499] Master 127.31.139.126:38193 was elected leader, sending a full tablet report...
I20260810 07:52:42.510164 511 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.66:37655
I20260810 07:52:42.510282 738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.66:37655 every 8 connection(s)
I20260810 07:52:42.513018 511 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
I20260810 07:52:42.522050 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 511
I20260810 07:52:42.522639 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/instance
I20260810 07:52:42.536649 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.67:0
--local_ip_for_outbound_sockets=127.31.139.67
--webserver_interface=127.31.139.67
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 07:52:42.569576 741 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
I20260810 07:52:42.569629 742 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:52:42.569813 740 heartbeater.cc:344] Connected to a master server at 127.31.139.124:36273
I20260810 07:52:42.570194 742 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:42.570187 741 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:42.570187 740 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:42.572008 740 heartbeater.cc:507] Master 127.31.139.124:36273 requested a full tablet report, sending...
I20260810 07:52:42.572023 741 heartbeater.cc:507] Master 127.31.139.125:46845 requested a full tablet report, sending...
I20260810 07:52:42.572068 742 heartbeater.cc:507] Master 127.31.139.126:38193 requested a full tablet report, sending...
I20260810 07:52:42.575686 32357 ts_manager.cc:194] Registered new tserver with Master: b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:42.575666 32586 ts_manager.cc:194] Registered new tserver with Master: b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:42.578480 32470 ts_manager.cc:194] Registered new tserver with Master: b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:42.581209 32470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.66:41637
I20260810 07:52:42.610294 546 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:42.611737 546 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
W20260810 07:52:42.896580 756 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:52:42.897374 756 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:42.898301 756 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:52:42.933068 756 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:42.934194 756 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.67
I20260810 07:52:42.973129 756 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.139.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.67
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:42.974606 756 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:42.976334 756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:52:42.990353 765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:43.084961 741 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:52:43.115787 776 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:52:43.116999 776 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:43.122000 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:43.122115 776 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:43.122229 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:43.124121 32331 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:43.125159 776 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:52:44.111214 780 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:52:44.111742 780 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:44.114552 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:44.122174 780 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:44.127238 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:44.128770 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:44.130525 781 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260810 07:52:42.991240 767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:44.384022 772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:52:44.386368 771 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1391 milliseconds
I20260810 07:52:44.386487 756 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:52:44.387895 756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:52:44.390216 756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:52:44.391600 756 hybrid_clock.cc:648] HybridClock initialized: now 1786348364391576 us; error 76 us; skew 500 ppm
I20260810 07:52:44.392596 756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:52:44.400022 756 webserver.cc:533] Webserver started at http://127.31.139.67:33409/ using document root <none> and password file <none>
I20260810 07:52:44.401129 756 fs_manager.cc:362] Metadata directory not provided
I20260810 07:52:44.401417 756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:52:44.401957 756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:52:44.406911 756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data/instance:
uuid: "9e545e0597e24d61b7265abed2ccdcfb"
format_stamp: "Formatted at 2026-08-10 07:52:44 on dist-test-slave-x5fp"
I20260810 07:52:44.408056 756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/instance:
uuid: "9e545e0597e24d61b7265abed2ccdcfb"
format_stamp: "Formatted at 2026-08-10 07:52:44 on dist-test-slave-x5fp"
I20260810 07:52:44.417109 756 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.010s sys 0.001s
I20260810 07:52:44.423271 791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:44.424449 756 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260810 07:52:44.424844 756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal
uuid: "9e545e0597e24d61b7265abed2ccdcfb"
format_stamp: "Formatted at 2026-08-10 07:52:44 on dist-test-slave-x5fp"
I20260810 07:52:44.425172 756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:52:44.486475 756 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:52:44.488189 756 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:52:44.488677 756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:52:44.491461 756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:52:44.495959 756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:52:44.496186 756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:44.496519 756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:52:44.496681 756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:52:44.653784 756 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.67:42917
I20260810 07:52:44.653859 959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.67:42917 every 8 connection(s)
I20260810 07:52:44.656474 756 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
I20260810 07:52:44.661437 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 756
I20260810 07:52:44.661913 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/instance
I20260810 07:52:44.684609 978 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:52:44.685572 978 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:44.689821 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:52:44.690052 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:44.688234 978 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:52:44.691896 32338 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:44.693532 978 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:52:44.707544 961 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
I20260810 07:52:44.708065 961 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:44.709444 961 heartbeater.cc:507] Master 127.31.139.125:46845 requested a full tablet report, sending...
I20260810 07:52:44.712245 32470 ts_manager.cc:194] Registered new tserver with Master: 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:44.713856 32470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.67:54007
I20260810 07:52:44.716435 963 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:52:44.716861 963 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:44.717415 960 heartbeater.cc:344] Connected to a master server at 127.31.139.124:36273
I20260810 07:52:44.717720 960 heartbeater.cc:461] Registering TS with master...
I20260810 07:52:44.717888 963 heartbeater.cc:507] Master 127.31.139.126:38193 requested a full tablet report, sending...
I20260810 07:52:44.718617 960 heartbeater.cc:507] Master 127.31.139.124:36273 requested a full tablet report, sending...
I20260810 07:52:44.721117 32586 ts_manager.cc:194] Registered new tserver with Master: 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:44.721261 32356 ts_manager.cc:194] Registered new tserver with Master: 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:44.742296 32301 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260810 07:52:44.759305 986 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:52:44.760054 986 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:52:44.762493 986 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:52:44.763403 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:52:44.763660 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:52:44.765934 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:52:44.769717 986 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:52:44.807147 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_19e302a19ee345b69a46f08da4f01e2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:44.808254 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e4da9efb16ad42f28387d322f7939c5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:44.808810 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b63a18a5bbad48a2a3b5436c4c2816b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:44.810923 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:44.811318 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b63a18a5bbad48a2a3b5436c4c2816b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:44.811296 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4da9efb16ad42f28387d322f7939c5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:44.811280 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19e302a19ee345b69a46f08da4f01e2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:44.813090 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:44.920822 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 32311
I20260810 07:52:44.940982 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b63a18a5bbad48a2a3b5436c4c2816b8" } new_table_name: "default.Table_513fd06005a64f3dabb9c2f6a52b5efe" modify_external_catalogs: true
I20260810 07:52:44.965947 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_e4da9efb16ad42f28387d322f7939c5a" } new_table_name: "default.Table_57b981a00fea4dfda77bba0d8c1dcfff" modify_external_catalogs: true
W20260810 07:52:44.987766 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.139.126:38193 (error 108) [suppressed 6 similar messages]
W20260810 07:52:44.988816 500 heartbeater.cc:646] Failed to heartbeat to 127.31.139.126:38193 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.31.139.126:38193 (error 108)
W20260810 07:52:45.030126 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:45.045884 385 tablet_service.cc:1511] Processing CreateTablet for tablet 0075243274794151ad4e2e0130ae04ef (DEFAULT_TABLE table=default.Table_b63a18a5bbad48a2a3b5436c4c2816b8 [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.050486 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0075243274794151ad4e2e0130ae04ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.053032 646 tablet_service.cc:1511] Processing CreateTablet for tablet 0075243274794151ad4e2e0130ae04ef (DEFAULT_TABLE table=default.Table_b63a18a5bbad48a2a3b5436c4c2816b8 [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.055357 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0075243274794151ad4e2e0130ae04ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.075050 872 tablet_service.cc:1511] Processing CreateTablet for tablet 0075243274794151ad4e2e0130ae04ef (DEFAULT_TABLE table=default.Table_b63a18a5bbad48a2a3b5436c4c2816b8 [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.078488 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0075243274794151ad4e2e0130ae04ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.092134 1047 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:45.095295 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_19e302a19ee345b69a46f08da4f01e2f" } modify_external_catalogs: true
I20260810 07:52:45.095988 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_19e302a19ee345b69a46f08da4f01e2f" } modify_external_catalogs: true
I20260810 07:52:45.107829 32470 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4" } modify_external_catalogs: true
I20260810 07:52:45.108410 32470 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4" } modify_external_catalogs: true
I20260810 07:52:45.111335 1047 tablet_bootstrap.cc:654] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.119067 1047 log.cc:826] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Log is configured to *not* fsync() on all Append() calls
W20260810 07:52:45.123283 742 heartbeater.cc:646] Failed to heartbeat to 127.31.139.126:38193 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
W20260810 07:52:45.135607 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0075243274794151ad4e2e0130ae04ef consensus state has no leader
I20260810 07:52:45.136114 1047 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:45.136879 1047 ts_tablet_manager.cc:1403] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.045s user 0.013s sys 0.010s
W20260810 07:52:45.137647 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 026bcdbffe74424b8f05834647036e7b has no consensus state
I20260810 07:52:45.147783 32467 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet aa7cddc9124f4054ad9f50453f69e0f5
I20260810 07:52:45.152264 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f3ba52251f854997b504a1640240b5a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.154340 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3ba52251f854997b504a1640240b5a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.166108 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_efcdf13d05104f71aa1f595ee2834d3c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.168644 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efcdf13d05104f71aa1f595ee2834d3c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.169312 1047 raft_consensus.cc:359] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.170323 1047 raft_consensus.cc:385] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.170658 1047 raft_consensus.cc:740] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.171751 1047 consensus_queue.cc:260] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.177253 384 tablet_service.cc:1511] Processing CreateTablet for tablet 026bcdbffe74424b8f05834647036e7b (DEFAULT_TABLE table=default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.178576 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bcdbffe74424b8f05834647036e7b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:45.181459 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0075243274794151ad4e2e0130ae04ef consensus state has no leader
I20260810 07:52:45.184252 871 tablet_service.cc:1511] Processing CreateTablet for tablet 026bcdbffe74424b8f05834647036e7b (DEFAULT_TABLE table=default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.185681 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bcdbffe74424b8f05834647036e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.192351 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e8025ef817e04dd8a20bd60b2c8893df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.194173 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8025ef817e04dd8a20bd60b2c8893df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.195856 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b4ff0f7881d746e9868e01b735277981"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.197824 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4ff0f7881d746e9868e01b735277981 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.198940 646 tablet_service.cc:1511] Processing CreateTablet for tablet 026bcdbffe74424b8f05834647036e7b (DEFAULT_TABLE table=default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.200075 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bcdbffe74424b8f05834647036e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.202538 382 tablet_service.cc:1511] Processing CreateTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 (DEFAULT_TABLE table=default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.203899 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f46f1ca4dbe4d2c8ead06651437e042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.217760 645 tablet_service.cc:1511] Processing CreateTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 (DEFAULT_TABLE table=default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.219144 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f46f1ca4dbe4d2c8ead06651437e042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.219360 869 tablet_service.cc:1511] Processing CreateTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 (DEFAULT_TABLE table=default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.220566 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f46f1ca4dbe4d2c8ead06651437e042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.221617 961 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:52:45.241422 1047 ts_tablet_manager.cc:1434] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.104s user 0.036s sys 0.026s
I20260810 07:52:45.259915 32470 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:45.263892 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:45.263880 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:45.266322 644 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:45.284749 1047 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:45.296446 1047 tablet_bootstrap.cc:654] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:45.302024 755 tablet.cc:2406] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:52:45.323832 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0f46f1ca4dbe4d2c8ead06651437e042 because the tablet was not found. No further retry: Not found: Tablet not found: 0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:45.323915 1064 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:45.326290 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 0f46f1ca4dbe4d2c8ead06651437e042 because the tablet was not found. No further retry: Not found: Tablet not found: 0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:45.329686 1071 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:45.338995 1064 tablet_bootstrap.cc:654] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.341856 1064 log.cc:826] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:52:45.342346 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
I20260810 07:52:45.343910 645 tablet_service.cc:1511] Processing CreateTablet for tablet f176f0909d064a93ab2384ac037123df (DEFAULT_TABLE table=default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.345182 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f176f0909d064a93ab2384ac037123df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.351818 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b4ff0f7881d746e9868e01b735277981" } new_table_name: "default.Table_74217fffb3f841cb8d27127268ccdc88" modify_external_catalogs: true
W20260810 07:52:45.353164 963 heartbeater.cc:646] Failed to heartbeat to 127.31.139.126:38193 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
W20260810 07:52:45.357018 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 0f46f1ca4dbe4d2c8ead06651437e042 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f46f1ca4dbe4d2c8ead06651437e042 already in progress: creating tablet
I20260810 07:52:45.339044 385 tablet_service.cc:1511] Processing CreateTablet for tablet f176f0909d064a93ab2384ac037123df (DEFAULT_TABLE table=default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.358425 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f176f0909d064a93ab2384ac037123df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.358623 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_57b981a00fea4dfda77bba0d8c1dcfff" } modify_external_catalogs: true
I20260810 07:52:45.359200 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_57b981a00fea4dfda77bba0d8c1dcfff" } modify_external_catalogs: true
I20260810 07:52:45.362119 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_e8025ef817e04dd8a20bd60b2c8893df" } new_table_name: "default.Table_d256f45e55724b6091b827c2d0119442" modify_external_catalogs: true
I20260810 07:52:45.363608 1062 raft_consensus.cc:493] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:45.364012 872 tablet_service.cc:1511] Processing CreateTablet for tablet f176f0909d064a93ab2384ac037123df (DEFAULT_TABLE table=default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.364262 1062 raft_consensus.cc:515] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.365828 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f176f0909d064a93ab2384ac037123df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.379774 1062 leader_election.cc:290] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:45.384439 1071 tablet_bootstrap.cc:654] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:45.401203 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3f679d04625b4c0690a4ce8ea2a5e87b has no consensus state
I20260810 07:52:45.403396 1064 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:45.404553 1064 ts_tablet_manager.cc:1403] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.081s user 0.013s sys 0.014s
I20260810 07:52:45.407043 1047 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:45.408389 1047 ts_tablet_manager.cc:1403] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.124s user 0.013s sys 0.003s
I20260810 07:52:45.413938 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_361112e9a662427ca1c92cdf908e1ff6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:45.411679 1047 raft_consensus.cc:359] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:45.430589 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6f37bba532b4d8bbc00e6d91f4c1820 has no consensus state
I20260810 07:52:45.429371 1047 raft_consensus.cc:385] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:52:45.436373 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_361112e9a662427ca1c92cdf908e1ff6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.388091 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:45.442502 573 leader_election.cc:343] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:45.451431 1071 log.cc:826] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Log is configured to *not* fsync() on all Append() calls
I20260810 07:52:45.435516 1047 raft_consensus.cc:740] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.457062 1047 consensus_queue.cc:260] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.459196 1064 raft_consensus.cc:359] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.460639 1064 raft_consensus.cc:385] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.461122 1064 raft_consensus.cc:740] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.462899 1064 consensus_queue.cc:260] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.465907 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:45.466789 897 raft_consensus.cc:2468] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:45.468595 574 leader_election.cc:304] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:45.469900 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_efcdf13d05104f71aa1f595ee2834d3c" } modify_external_catalogs: true
I20260810 07:52:45.470562 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_efcdf13d05104f71aa1f595ee2834d3c" } modify_external_catalogs: true
I20260810 07:52:45.474556 1064 ts_tablet_manager.cc:1434] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.067s user 0.047s sys 0.006s
I20260810 07:52:45.481367 1045 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:52:45.482112 1045 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
I20260810 07:52:45.486692 1064 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:45.494565 1062 raft_consensus.cc:2804] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:45.495024 1062 raft_consensus.cc:493] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:45.495505 1062 raft_consensus.cc:3060] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:45.502177 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
I20260810 07:52:45.511404 645 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5eed268ee545ee91420cd792608b23 (DEFAULT_TABLE table=default.Table_f3ba52251f854997b504a1640240b5a8 [id=2e56b185007f4d798dba6e17d0b14818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.512944 869 tablet_service.cc:1511] Processing CreateTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (DEFAULT_TABLE table=default.Table_d256f45e55724b6091b827c2d0119442 [id=a1b55d4a058a4d43bbbbce82198aa6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.513151 872 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5eed268ee545ee91420cd792608b23 (DEFAULT_TABLE table=default.Table_f3ba52251f854997b504a1640240b5a8 [id=2e56b185007f4d798dba6e17d0b14818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.514345 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5eed268ee545ee91420cd792608b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.514739 382 tablet_service.cc:1511] Processing CreateTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (DEFAULT_TABLE table=default.Table_d256f45e55724b6091b827c2d0119442 [id=a1b55d4a058a4d43bbbbce82198aa6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.515887 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f37bba532b4d8bbc00e6d91f4c1820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.518204 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f37bba532b4d8bbc00e6d91f4c1820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.521247 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5eed268ee545ee91420cd792608b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.523262 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_753b8ca312194a6d9e028c358cbe2007"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.525058 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_753b8ca312194a6d9e028c358cbe2007 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.526046 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
I20260810 07:52:45.538522 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 026bcdbffe74424b8f05834647036e7b
I20260810 07:52:45.539321 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_39969fe570e440f2a2edc5187b76e63b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.541532 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39969fe570e440f2a2edc5187b76e63b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.519208 385 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5eed268ee545ee91420cd792608b23 (DEFAULT_TABLE table=default.Table_f3ba52251f854997b504a1640240b5a8 [id=2e56b185007f4d798dba6e17d0b14818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.545928 871 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:45.546963 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:45.547984 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5eed268ee545ee91420cd792608b23. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:45.549415 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 026bcdbffe74424b8f05834647036e7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026bcdbffe74424b8f05834647036e7b already in progress: creating tablet
I20260810 07:52:45.553733 382 tablet_service.cc:1511] Processing CreateTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (DEFAULT_TABLE table=default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.554932 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f679d04625b4c0690a4ce8ea2a5e87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.564690 1071 tablet_bootstrap.cc:492] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:45.565395 1071 ts_tablet_manager.cc:1403] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.236s user 0.014s sys 0.005s
I20260810 07:52:45.567852 871 tablet_service.cc:1511] Processing CreateTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (DEFAULT_TABLE table=default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.568082 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_672ddf2b579f43f9976d524307dac803"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.580945 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_672ddf2b579f43f9976d524307dac803 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:45.592545 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 026bcdbffe74424b8f05834647036e7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026bcdbffe74424b8f05834647036e7b already in progress: creating tablet
I20260810 07:52:45.593735 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet f176f0909d064a93ab2384ac037123df
I20260810 07:52:45.595070 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f679d04625b4c0690a4ce8ea2a5e87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.598971 868 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:45.601454 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet f176f0909d064a93ab2384ac037123df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f176f0909d064a93ab2384ac037123df already in progress: creating tablet
I20260810 07:52:45.602195 382 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:45.603998 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet f176f0909d064a93ab2384ac037123df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f176f0909d064a93ab2384ac037123df already in progress: creating tablet
I20260810 07:52:45.548663 644 tablet_service.cc:1511] Processing CreateTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (DEFAULT_TABLE table=default.Table_d256f45e55724b6091b827c2d0119442 [id=a1b55d4a058a4d43bbbbce82198aa6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.555998 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:45.608898 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 026bcdbffe74424b8f05834647036e7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 026bcdbffe74424b8f05834647036e7b already in progress: creating tablet
I20260810 07:52:45.573136 642 tablet_service.cc:1511] Processing CreateTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (DEFAULT_TABLE table=default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.597506 639 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:45.613376 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet f176f0909d064a93ab2384ac037123df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f176f0909d064a93ab2384ac037123df already in progress: creating tablet
I20260810 07:52:45.614547 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f37bba532b4d8bbc00e6d91f4c1820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.618120 1064 tablet_bootstrap.cc:700] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent opening tablet: real 0.118s user 0.002s sys 0.000s
I20260810 07:52:45.619196 1064 tablet_bootstrap.cc:654] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.621232 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f679d04625b4c0690a4ce8ea2a5e87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.601644 1062 raft_consensus.cc:515] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.606671 1047 ts_tablet_manager.cc:1434] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.198s user 0.008s sys 0.000s
I20260810 07:52:45.644649 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:45.646162 573 leader_election.cc:343] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:45.668118 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_5f98a4e3c55c44ec96262a12eb5cff99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:45.669579 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_361112e9a662427ca1c92cdf908e1ff6" } new_table_name: "default.Table_612e3c4860664a5a964122487ce93bed" modify_external_catalogs: true
W20260810 07:52:45.670056 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f98a4e3c55c44ec96262a12eb5cff99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.672981 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f3ba52251f854997b504a1640240b5a8" } new_table_name: "default.Table_1992a5b4d5144b85986cec15a6b24446" modify_external_catalogs: true
I20260810 07:52:45.679497 1047 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:45.689970 1071 raft_consensus.cc:359] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.690902 1071 raft_consensus.cc:385] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.691218 1071 raft_consensus.cc:740] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.691463 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:45.692348 1068 tablet_replica.cc:333] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:45.693253 1068 raft_consensus.cc:2243] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:45.693836 1068 raft_consensus.cc:2272] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:45.697566 1068 ts_tablet_manager.cc:1916] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:45.697170 1071 consensus_queue.cc:260] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.707491 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:45.708081 897 raft_consensus.cc:3060] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:45.709090 1062 leader_election.cc:290] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:45.716219 967 tablet.cc:2406] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:52:45.722224 897 raft_consensus.cc:2468] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:45.723704 574 leader_election.cc:304] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:45.726122 1062 raft_consensus.cc:2804] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:45.735317 869 tablet_service.cc:1511] Processing CreateTablet for tablet e13d8a5248d54b00aa4a47874561745e (DEFAULT_TABLE table=default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.736507 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13d8a5248d54b00aa4a47874561745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.737526 872 tablet_service.cc:1511] Processing CreateTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 (DEFAULT_TABLE table=default.Table_361112e9a662427ca1c92cdf908e1ff6 [id=0d62e87e4971408faa84ce2850fe01e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.739975 1064 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:45.740556 1064 ts_tablet_manager.cc:1403] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.254s user 0.008s sys 0.008s
I20260810 07:52:45.741269 385 tablet_service.cc:1511] Processing CreateTablet for tablet e13d8a5248d54b00aa4a47874561745e (DEFAULT_TABLE table=default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.741683 1047 tablet_bootstrap.cc:654] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.742429 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13d8a5248d54b00aa4a47874561745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.745096 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d1bfe95e5d4d8697584b05b28339f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.745684 382 tablet_service.cc:1511] Processing CreateTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 (DEFAULT_TABLE table=default.Table_361112e9a662427ca1c92cdf908e1ff6 [id=0d62e87e4971408faa84ce2850fe01e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.746767 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d1bfe95e5d4d8697584b05b28339f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.753253 642 tablet_service.cc:1511] Processing CreateTablet for tablet e13d8a5248d54b00aa4a47874561745e (DEFAULT_TABLE table=default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.754540 1062 raft_consensus.cc:697] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:45.758291 1062 consensus_queue.cc:237] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.763401 644 tablet_service.cc:1511] Processing CreateTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 (DEFAULT_TABLE table=default.Table_361112e9a662427ca1c92cdf908e1ff6 [id=0d62e87e4971408faa84ce2850fe01e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:45.764588 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d1bfe95e5d4d8697584b05b28339f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.768901 32470 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_672ddf2b579f43f9976d524307dac803" } modify_external_catalogs: true
I20260810 07:52:45.769805 32470 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_672ddf2b579f43f9976d524307dac803" } modify_external_catalogs: true
I20260810 07:52:45.743511 1064 raft_consensus.cc:359] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.781097 1064 raft_consensus.cc:385] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.781553 1064 raft_consensus.cc:740] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.782306 1064 consensus_queue.cc:260] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.784942 1064 ts_tablet_manager.cc:1434] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.044s user 0.004s sys 0.004s
I20260810 07:52:45.787931 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_34815222bcfd41c280ae4727db272f01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.789906 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34815222bcfd41c280ae4727db272f01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.792264 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:45.799572 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13d8a5248d54b00aa4a47874561745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:45.800983 32469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_39969fe570e440f2a2edc5187b76e63b" } modify_external_catalogs: true
I20260810 07:52:45.801605 32469 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_39969fe570e440f2a2edc5187b76e63b" } modify_external_catalogs: true
I20260810 07:52:45.803719 1065 tablet_replica.cc:333] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:45.804632 1065 raft_consensus.cc:2243] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:45.805276 1065 raft_consensus.cc:2272] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:45.808967 1047 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:45.809332 1065 ts_tablet_manager.cc:1916] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:45.809868 1071 ts_tablet_manager.cc:1434] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.241s user 0.030s sys 0.011s
I20260810 07:52:45.809448 1047 ts_tablet_manager.cc:1403] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.130s user 0.009s sys 0.002s
W20260810 07:52:45.824440 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99d1bfe95e5d4d8697584b05b28339f3 consensus state has no leader
I20260810 07:52:45.829330 1068 ts_tablet_manager.cc:1929] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:45.829838 1068 log.cc:1199] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/026bcdbffe74424b8f05834647036e7b
I20260810 07:52:45.830022 1071 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:45.830773 1068 ts_tablet_manager.cc:1950] T 026bcdbffe74424b8f05834647036e7b P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:45.834913 1047 raft_consensus.cc:359] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.835662 1047 raft_consensus.cc:385] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.836000 1047 raft_consensus.cc:740] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.841282 1064 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:45.842164 1071 tablet_bootstrap.cc:654] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:45.854192 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b5eed268ee545ee91420cd792608b23 consensus state has no leader
I20260810 07:52:45.855932 1047 consensus_queue.cc:260] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.860556 1064 tablet_bootstrap.cc:654] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.867317 1065 ts_tablet_manager.cc:1929] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:45.867686 1047 ts_tablet_manager.cc:1434] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.051s user 0.004s sys 0.002s
I20260810 07:52:45.867837 1065 log.cc:1199] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/026bcdbffe74424b8f05834647036e7b
I20260810 07:52:45.868851 1065 ts_tablet_manager.cc:1950] T 026bcdbffe74424b8f05834647036e7b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:45.872596 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 026bcdbffe74424b8f05834647036e7b (table default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]) successfully deleted
I20260810 07:52:45.873400 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 026bcdbffe74424b8f05834647036e7b (table default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]) successfully deleted
I20260810 07:52:45.883004 1047 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:45.885803 1064 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:45.886282 1064 ts_tablet_manager.cc:1403] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.006s
I20260810 07:52:45.889195 1064 raft_consensus.cc:359] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.889919 1064 raft_consensus.cc:385] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.890206 1064 raft_consensus.cc:740] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.895315 1071 tablet_bootstrap.cc:492] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:45.895798 1071 ts_tablet_manager.cc:1403] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.006s
I20260810 07:52:45.896375 32470 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 247b124b61a24d2a9f38726cd96a064c
I20260810 07:52:45.898554 1071 raft_consensus.cc:359] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.899284 1071 raft_consensus.cc:385] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:45.899551 1071 raft_consensus.cc:740] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:45.900451 1071 consensus_queue.cc:260] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.902930 1047 tablet_bootstrap.cc:654] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:45.913497 1071 ts_tablet_manager.cc:1434] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.017s user 0.005s sys 0.002s
I20260810 07:52:45.920599 1081 raft_consensus.cc:493] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:45.921478 1081 raft_consensus.cc:515] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:45.925586 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_60a9e92ff9114091984968ca5c73699c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.928179 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60a9e92ff9114091984968ca5c73699c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.930042 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f4269fe23ea14f68b3bbf0770396ffc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.932185 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4269fe23ea14f68b3bbf0770396ffc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.906267 1064 consensus_queue.cc:260] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:45.937090 1081 leader_election.cc:290] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:45.943198 1064 ts_tablet_manager.cc:1434] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.057s user 0.010s sys 0.008s
I20260810 07:52:45.944654 1064 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:45.945902 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:45.946659 1065 tablet_replica.cc:333] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:45.947371 1065 raft_consensus.cc:2243] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:45.947881 1065 raft_consensus.cc:2272] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:45.952828 1065 ts_tablet_manager.cc:1916] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:45.971175 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b21be299a7664348b1a5fb51d9f031ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:45.973356 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b21be299a7664348b1a5fb51d9f031ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:45.984661 1071 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:45.993654 1064 tablet_bootstrap.cc:654] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.003715 1065 ts_tablet_manager.cc:1929] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:46.004146 1065 log.cc:1199] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:46.013441 1065 ts_tablet_manager.cc:1950] T 0f46f1ca4dbe4d2c8ead06651437e042 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:46.016006 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:46.017148 897 raft_consensus.cc:2393] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:46.017711 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 0f46f1ca4dbe4d2c8ead06651437e042 (table default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]) successfully deleted
I20260810 07:52:46.023046 1071 tablet_bootstrap.cc:654] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.026543 1064 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.026990 1064 ts_tablet_manager.cc:1403] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.002s
I20260810 07:52:46.029824 1064 raft_consensus.cc:359] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.030048 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_5f98a4e3c55c44ec96262a12eb5cff99" } new_table_name: "default.Table_f77039cfd411423da5e7bf5e02a6aa81" modify_external_catalogs: true
I20260810 07:52:46.030525 1064 raft_consensus.cc:385] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.030797 1064 raft_consensus.cc:740] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:52:46.032822 501 tablet.cc:2406] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:52:46.034410 1064 consensus_queue.cc:260] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.035794 1047 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:46.036249 1047 ts_tablet_manager.cc:1403] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.153s user 0.033s sys 0.055s
I20260810 07:52:46.039073 1047 raft_consensus.cc:359] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.039768 1047 raft_consensus.cc:385] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.040042 1047 raft_consensus.cc:740] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.042479 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0075243274794151ad4e2e0130ae04ef" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.044919 308 leader_election.cc:304] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:46.046311 1081 raft_consensus.cc:3060] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.049566 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
I20260810 07:52:46.052604 1064 ts_tablet_manager.cc:1434] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.025s user 0.005s sys 0.002s
I20260810 07:52:46.053562 1081 raft_consensus.cc:2749] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:46.053686 1064 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.040766 1047 consensus_queue.cc:260] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.057168 1047 ts_tablet_manager.cc:1434] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.021s user 0.005s sys 0.003s
I20260810 07:52:46.055848 32468 catalog_manager.cc:5719] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:46.058152 1047 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:46.064519 1047 tablet_bootstrap.cc:654] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.065594 385 tablet_service.cc:1511] Processing CreateTablet for tablet d7626d30f4574658bb226f2e915ee0de (DEFAULT_TABLE table=default.Table_5f98a4e3c55c44ec96262a12eb5cff99 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.066785 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7626d30f4574658bb226f2e915ee0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.070484 642 tablet_service.cc:1511] Processing CreateTablet for tablet d7626d30f4574658bb226f2e915ee0de (DEFAULT_TABLE table=default.Table_5f98a4e3c55c44ec96262a12eb5cff99 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.071990 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7626d30f4574658bb226f2e915ee0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.073132 869 tablet_service.cc:1511] Processing CreateTablet for tablet d7626d30f4574658bb226f2e915ee0de (DEFAULT_TABLE table=default.Table_5f98a4e3c55c44ec96262a12eb5cff99 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.073751 32468 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
I20260810 07:52:46.074237 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7626d30f4574658bb226f2e915ee0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.081935 872 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:46.082780 1065 tablet_replica.cc:333] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:46.082489 32464 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_34815222bcfd41c280ae4727db272f01" } new_table_name: "default.Table_aeb2755859d043bfa96d14a40dddb4be" modify_external_catalogs: true
I20260810 07:52:46.083515 1065 raft_consensus.cc:2243] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:46.083994 1065 raft_consensus.cc:2272] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:46.086650 1065 ts_tablet_manager.cc:1916] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:46.087648 382 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4772c3359a426e9f105760afff40cb (DEFAULT_TABLE table=default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.089252 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4772c3359a426e9f105760afff40cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.090834 872 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4772c3359a426e9f105760afff40cb (DEFAULT_TABLE table=default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.091934 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4772c3359a426e9f105760afff40cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.092531 644 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4772c3359a426e9f105760afff40cb (DEFAULT_TABLE table=default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.094097 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4772c3359a426e9f105760afff40cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.108887 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_60a9e92ff9114091984968ca5c73699c" } modify_external_catalogs: true
I20260810 07:52:46.109930 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_60a9e92ff9114091984968ca5c73699c" } modify_external_catalogs: true
I20260810 07:52:46.122910 1071 tablet_bootstrap.cc:492] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:46.123536 1071 ts_tablet_manager.cc:1403] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.157s user 0.005s sys 0.009s
I20260810 07:52:46.127441 1071 raft_consensus.cc:359] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.138199 1064 tablet_bootstrap.cc:654] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.128443 1071 raft_consensus.cc:385] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.144953 1071 raft_consensus.cc:740] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.146169 1071 consensus_queue.cc:260] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.151798 1071 ts_tablet_manager.cc:1434] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.028s user 0.006s sys 0.005s
W20260810 07:52:46.158946 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d7626d30f4574658bb226f2e915ee0de consensus state has no leader
I20260810 07:52:46.165867 1071 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:46.159135 1065 ts_tablet_manager.cc:1929] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:46.166970 1065 log.cc:1199] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/f176f0909d064a93ab2384ac037123df
I20260810 07:52:46.167724 1065 ts_tablet_manager.cc:1950] T f176f0909d064a93ab2384ac037123df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:46.170517 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet f176f0909d064a93ab2384ac037123df (table default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]) successfully deleted
I20260810 07:52:46.180438 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_ae672329134f4bbf8959c4cc2a85982c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.182274 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae672329134f4bbf8959c4cc2a85982c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.187918 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 0b4772c3359a426e9f105760afff40cb
I20260810 07:52:46.191412 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:46.191923 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:46.192010 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:46.193423 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0b4772c3359a426e9f105760afff40cb because the tablet was not found. No further retry: Not found: Tablet not found: 0b4772c3359a426e9f105760afff40cb
W20260810 07:52:46.194769 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 0b4772c3359a426e9f105760afff40cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b4772c3359a426e9f105760afff40cb already in progress: creating tablet
I20260810 07:52:46.200533 1047 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:46.201030 1047 ts_tablet_manager.cc:1403] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.143s user 0.060s sys 0.068s
I20260810 07:52:46.204263 1047 raft_consensus.cc:359] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.204996 1047 raft_consensus.cc:385] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.205266 1047 raft_consensus.cc:740] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.205994 1047 consensus_queue.cc:260] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.207559 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_513fd06005a64f3dabb9c2f6a52b5efe [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]). Expected version 1 got 0
W20260810 07:52:46.209576 32464 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9369baf30ab64646b7bc042c6da1987b has no consensus state
I20260810 07:52:46.212437 384 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bcdbffe74424b8f05834647036e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:46.215593 1047 ts_tablet_manager.cc:1434] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.014s user 0.007s sys 0.001s
W20260810 07:52:46.219460 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d7626d30f4574658bb226f2e915ee0de consensus state has no leader
I20260810 07:52:46.219719 1047 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:46.225266 1064 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.225755 1064 ts_tablet_manager.cc:1403] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.172s user 0.016s sys 0.006s
W20260810 07:52:46.226341 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 0b4772c3359a426e9f105760afff40cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b4772c3359a426e9f105760afff40cb already in progress: creating tablet
I20260810 07:52:46.228061 1103 tablet_replica.cc:333] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:46.228582 1064 raft_consensus.cc:359] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.229308 1064 raft_consensus.cc:385] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.229627 1064 raft_consensus.cc:740] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.231808 1047 tablet_bootstrap.cc:654] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.233697 1064 consensus_queue.cc:260] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.237816 1062 raft_consensus.cc:493] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.238271 1062 raft_consensus.cc:515] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.240391 1062 leader_election.cc:290] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:46.242256 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f46f1ca4dbe4d2c8ead06651437e042" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.242954 417 raft_consensus.cc:2468] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:46.243988 1081 raft_consensus.cc:493] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:52:46.243928 574 leader_election.cc:343] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:46.244450 1081 raft_consensus.cc:515] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.245553 573 leader_election.cc:304] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:46.246629 1062 raft_consensus.cc:2804] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:46.246968 1062 raft_consensus.cc:493] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:46.247133 1081 leader_election.cc:290] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.247291 1062 raft_consensus.cc:3060] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:46.250325 308 leader_election.cc:343] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 026bcdbffe74424b8f05834647036e7b
I20260810 07:52:46.251652 1064 ts_tablet_manager.cc:1434] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
W20260810 07:52:46.252841 309 leader_election.cc:343] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 026bcdbffe74424b8f05834647036e7b
I20260810 07:52:46.253106 1103 raft_consensus.cc:2243] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:46.253252 309 leader_election.cc:304] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:46.254073 1103 raft_consensus.cc:2272] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:46.258525 1081 raft_consensus.cc:2706] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260810 07:52:46.259341 1062 raft_consensus.cc:515] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.262496 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f46f1ca4dbe4d2c8ead06651437e042" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:46.263159 417 raft_consensus.cc:3060] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.264477 1064 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:46.264786 574 leader_election.cc:343] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:46.265753 1062 leader_election.cc:290] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:46.267580 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_792d89d794dd460e8a64665f11bd3641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.270107 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_792d89d794dd460e8a64665f11bd3641 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.273033 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_162cd65e2eaa48c195c35cbf6ce89c8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:46.273912 1064 tablet_bootstrap.cc:654] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:46.274816 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_162cd65e2eaa48c195c35cbf6ce89c8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.275597 1047 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:46.276073 1047 ts_tablet_manager.cc:1403] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.057s user 0.014s sys 0.002s
I20260810 07:52:46.278967 1047 raft_consensus.cc:359] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.279654 1047 raft_consensus.cc:385] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.279944 1047 raft_consensus.cc:740] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.280859 1047 consensus_queue.cc:260] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.282222 417 raft_consensus.cc:2468] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:46.286743 573 leader_election.cc:304] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:46.288836 1071 tablet_bootstrap.cc:654] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.293061 1062 raft_consensus.cc:2804] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:46.298509 1047 ts_tablet_manager.cc:1434] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.004s sys 0.004s
I20260810 07:52:46.298714 1064 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.299220 1064 ts_tablet_manager.cc:1403] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.002s
I20260810 07:52:46.290468 1103 ts_tablet_manager.cc:1916] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:46.301869 1064 raft_consensus.cc:359] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.302559 1064 raft_consensus.cc:385] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.302825 1064 raft_consensus.cc:740] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.303733 1064 consensus_queue.cc:260] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.308934 1047 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:46.314913 1062 raft_consensus.cc:697] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:46.315711 1064 ts_tablet_manager.cc:1434] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260810 07:52:46.315853 1062 consensus_queue.cc:237] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.319788 32465 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 7945457ecebc409996fb8700505a8ce7
I20260810 07:52:46.322947 1047 tablet_bootstrap.cc:654] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.323613 1064 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.330507 1064 tablet_bootstrap.cc:654] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.337824 1075 raft_consensus.cc:493] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.338346 1075 raft_consensus.cc:515] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.347822 1071 tablet_bootstrap.cc:492] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:46.350996 1071 ts_tablet_manager.cc:1403] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.185s user 0.013s sys 0.001s
I20260810 07:52:46.351593 1075 leader_election.cc:290] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.354758 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:46.355283 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:46.355074 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_937b2a00ea624497b31e38f446787d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.357307 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_937b2a00ea624497b31e38f446787d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.364212 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_513fd06005a64f3dabb9c2f6a52b5efe" } new_table_name: "default.Table_a4b3c1181096402598c18155b4383ecb" modify_external_catalogs: true
I20260810 07:52:46.379537 1071 raft_consensus.cc:359] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.380538 1071 raft_consensus.cc:385] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.380945 1071 raft_consensus.cc:740] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.386327 1071 consensus_queue.cc:260] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.387260 417 raft_consensus.cc:1275] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:46.390379 897 raft_consensus.cc:1275] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:46.392517 1071 ts_tablet_manager.cc:1434] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260810 07:52:46.393538 1071 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:46.399180 1062 consensus_queue.cc:1048] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:46.405648 1064 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.406111 1064 ts_tablet_manager.cc:1403] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.083s user 0.012s sys 0.000s
I20260810 07:52:46.406118 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:46.406692 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:46.408066 32469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_753b8ca312194a6d9e028c358cbe2007" } modify_external_catalogs: true
I20260810 07:52:46.408612 32469 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_753b8ca312194a6d9e028c358cbe2007" } modify_external_catalogs: true
I20260810 07:52:46.408595 1064 raft_consensus.cc:359] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.409246 1064 raft_consensus.cc:385] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.409523 1064 raft_consensus.cc:740] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.411298 1062 consensus_queue.cc:1048] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:46.415473 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d256f45e55724b6091b827c2d0119442" } new_table_name: "default.Table_6454dbcc43ab45ff8313cbf3e725d310" modify_external_catalogs: true
I20260810 07:52:46.426343 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5eed268ee545ee91420cd792608b23" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.427119 678 raft_consensus.cc:2468] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:46.429118 1064 consensus_queue.cc:260] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.429138 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5eed268ee545ee91420cd792608b23" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.431731 792 leader_election.cc:304] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:46.432796 1075 raft_consensus.cc:2804] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:46.433175 1075 raft_consensus.cc:493] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:46.433555 1075 raft_consensus.cc:3060] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.434562 1064 ts_tablet_manager.cc:1434] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.028s user 0.004s sys 0.002s
I20260810 07:52:46.438347 1064 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:46.441521 792 leader_election.cc:343] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.443734 1064 tablet_bootstrap.cc:654] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.444589 1075 raft_consensus.cc:515] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.446964 1075 leader_election.cc:290] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.450086 1047 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:46.450523 1047 ts_tablet_manager.cc:1403] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.142s user 0.012s sys 0.018s
I20260810 07:52:46.451651 1071 tablet_bootstrap.cc:654] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.453315 1047 raft_consensus.cc:359] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.454031 1047 raft_consensus.cc:385] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.455570 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5eed268ee545ee91420cd792608b23" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:46.454443 1047 raft_consensus.cc:740] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.463701 1064 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.464104 1064 ts_tablet_manager.cc:1403] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.004s
I20260810 07:52:46.466534 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 1 got 0
I20260810 07:52:46.456676 1047 consensus_queue.cc:260] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.469967 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_1992a5b4d5144b85986cec15a6b24446 [id=2e56b185007f4d798dba6e17d0b14818]). Expected version 1 got 0
I20260810 07:52:46.466337 1064 raft_consensus.cc:359] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.471912 1064 raft_consensus.cc:385] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.472172 1064 raft_consensus.cc:740] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.472927 1047 ts_tablet_manager.cc:1434] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
I20260810 07:52:46.472821 1064 consensus_queue.cc:260] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.478029 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5eed268ee545ee91420cd792608b23" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:46.478556 1047 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:46.478607 678 raft_consensus.cc:3060] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.485262 1064 ts_tablet_manager.cc:1434] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.021s user 0.001s sys 0.006s
I20260810 07:52:46.485729 872 tablet_service.cc:1511] Processing CreateTablet for tablet 9369baf30ab64646b7bc042c6da1987b (DEFAULT_TABLE table=default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.486745 1064 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.487095 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9369baf30ab64646b7bc042c6da1987b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.490859 382 tablet_service.cc:1511] Processing CreateTablet for tablet 9369baf30ab64646b7bc042c6da1987b (DEFAULT_TABLE table=default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.492208 1064 tablet_bootstrap.cc:654] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.492246 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9369baf30ab64646b7bc042c6da1987b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.499873 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:46.502025 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6f37bba532b4d8bbc00e6d91f4c1820 consensus state has no leader
I20260810 07:52:46.518672 869 tablet_service.cc:1511] Processing CreateTablet for tablet 5b76191c5bed41509344376f1810e7aa (DEFAULT_TABLE table=default.Table_f4269fe23ea14f68b3bbf0770396ffc5 [id=a66d245c7ec84e8080475db270e64746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.519917 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b76191c5bed41509344376f1810e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.523360 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet e13d8a5248d54b00aa4a47874561745e
I20260810 07:52:46.525370 385 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:46.526775 871 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:46.529768 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet e13d8a5248d54b00aa4a47874561745e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e13d8a5248d54b00aa4a47874561745e already in progress: creating tablet
I20260810 07:52:46.531579 1047 tablet_bootstrap.cc:654] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.537395 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 1 got 0
I20260810 07:52:46.535614 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f46f1ca4dbe4d2c8ead06651437e042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:46.538172 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_6454dbcc43ab45ff8313cbf3e725d310 [id=a1b55d4a058a4d43bbbbce82198aa6ef]). Expected version 2 got 0
I20260810 07:52:46.538689 1068 tablet_replica.cc:333] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:46.539515 1068 raft_consensus.cc:2243] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:46.540359 1068 pending_rounds.cc:70] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:46.535612 639 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:46.532043 644 tablet_service.cc:1511] Processing CreateTablet for tablet 9369baf30ab64646b7bc042c6da1987b (DEFAULT_TABLE table=default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.542205 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9369baf30ab64646b7bc042c6da1987b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.542882 385 tablet_service.cc:1511] Processing CreateTablet for tablet 5b76191c5bed41509344376f1810e7aa (DEFAULT_TABLE table=default.Table_f4269fe23ea14f68b3bbf0770396ffc5 [id=a66d245c7ec84e8080475db270e64746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.544034 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b76191c5bed41509344376f1810e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.540656 1068 pending_rounds.cc:77] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882908466569216 op_type: NO_OP noop_request { }
I20260810 07:52:46.547118 1064 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.547088 1068 raft_consensus.cc:2889] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:46.547541 1064 ts_tablet_manager.cc:1403] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.061s user 0.016s sys 0.001s
I20260810 07:52:46.547513 1068 raft_consensus.cc:2272] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:46.549958 1064 raft_consensus.cc:359] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.550621 1064 raft_consensus.cc:385] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.551005 1064 raft_consensus.cc:740] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.552071 1064 consensus_queue.cc:260] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.553815 639 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:46.553783 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a4b3c1181096402598c18155b4383ecb" } modify_external_catalogs: true
I20260810 07:52:46.554498 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a4b3c1181096402598c18155b4383ecb" } modify_external_catalogs: true
I20260810 07:52:46.555145 645 tablet_service.cc:1511] Processing CreateTablet for tablet 5b76191c5bed41509344376f1810e7aa (DEFAULT_TABLE table=default.Table_f4269fe23ea14f68b3bbf0770396ffc5 [id=a66d245c7ec84e8080475db270e64746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.556341 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b76191c5bed41509344376f1810e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.558686 1064 ts_tablet_manager.cc:1434] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260810 07:52:46.563755 1064 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.564072 1068 ts_tablet_manager.cc:1916] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:46.568878 792 leader_election.cc:343] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.576184 1103 ts_tablet_manager.cc:1929] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:46.576771 1103 log.cc:1199] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/026bcdbffe74424b8f05834647036e7b
I20260810 07:52:46.577332 1075 raft_consensus.cc:493] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.578173 1103 ts_tablet_manager.cc:1950] T 026bcdbffe74424b8f05834647036e7b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:46.577813 1075 raft_consensus.cc:515] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.579809 1075 leader_election.cc:290] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.586063 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13d8a5248d54b00aa4a47874561745e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.587988 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 0075243274794151ad4e2e0130ae04ef
I20260810 07:52:46.588538 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13d8a5248d54b00aa4a47874561745e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:46.590209 792 leader_election.cc:343] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:46.590862 792 leader_election.cc:343] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.591223 792 leader_election.cc:304] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:46.591917 1075 raft_consensus.cc:2749] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:46.592949 1064 tablet_bootstrap.cc:654] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.602531 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 0075243274794151ad4e2e0130ae04ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:46.605340 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 0075243274794151ad4e2e0130ae04ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:46.610086 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_7e88718860ff4246b3dcf39078c78463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:46.610214 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_52cc17edf52f44798e287e045fe08934"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.611842 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e88718860ff4246b3dcf39078c78463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:46.613777 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52cc17edf52f44798e287e045fe08934 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.615162 1064 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.616459 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 0075243274794151ad4e2e0130ae04ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:46.617331 1065 tablet_replica.cc:333] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:46.615588 1064 ts_tablet_manager.cc:1403] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.004s
I20260810 07:52:46.620582 1064 raft_consensus.cc:359] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.621188 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 026bcdbffe74424b8f05834647036e7b (table default.Table_57b981a00fea4dfda77bba0d8c1dcfff [id=8bf717babd6e471e82b67e6ac0a28b6e]) successfully deleted
I20260810 07:52:46.621389 1064 raft_consensus.cc:385] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.621781 1064 raft_consensus.cc:740] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.622229 1103 tablet_replica.cc:333] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:46.622676 1064 consensus_queue.cc:260] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.623055 1103 raft_consensus.cc:2243] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:46.623558 1103 raft_consensus.cc:2272] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:46.612610 678 raft_consensus.cc:2468] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:46.626102 792 leader_election.cc:304] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:46.627261 1075 raft_consensus.cc:2804] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:46.628417 1064 ts_tablet_manager.cc:1434] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260810 07:52:46.632218 1064 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.637739 1081 raft_consensus.cc:493] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.638252 1081 raft_consensus.cc:515] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.639230 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_452dd2adccb043c685e5eb8af81bd4db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:46.640311 1081 leader_election.cc:290] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:46.643785 309 leader_election.cc:343] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: f176f0909d064a93ab2384ac037123df
I20260810 07:52:46.646842 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f176f0909d064a93ab2384ac037123df" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.647537 678 raft_consensus.cc:2468] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:52:46.648780 308 leader_election.cc:304] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:46.649684 1081 raft_consensus.cc:2804] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:46.650007 1081 raft_consensus.cc:493] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:46.650332 1081 raft_consensus.cc:3060] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.655332 1075 raft_consensus.cc:697] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:46.656392 1075 consensus_queue.cc:237] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.663408 1064 tablet_bootstrap.cc:654] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.676017 1064 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.676440 1064 ts_tablet_manager.cc:1403] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.000s
I20260810 07:52:46.676360 1065 raft_consensus.cc:2243] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:46.678105 1142 raft_consensus.cc:493] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.678645 1142 raft_consensus.cc:515] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.678952 1064 raft_consensus.cc:359] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:46.679383 1065 pending_rounds.cc:70] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Trying to abort 2 pending ops.
I20260810 07:52:46.679543 1064 raft_consensus.cc:385] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.679805 1064 raft_consensus.cc:740] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.679716 1065 pending_rounds.cc:77] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316882908599865344 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "0075243274794151ad4e2e0130ae04ef" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_513fd06005a64f3dabb9c2f6a52b5efe" dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" new_extra_config { } }
I20260810 07:52:46.682062 1065 pending_rounds.cc:77] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882906158092288 op_type: NO_OP noop_request { }
I20260810 07:52:46.684892 1065 raft_consensus.cc:2889] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:46.685588 1065 raft_consensus.cc:2272] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:46.689563 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.691141 1103 ts_tablet_manager.cc:1916] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:46.693830 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_162cd65e2eaa48c195c35cbf6ce89c8a" } new_table_name: "default.Table_c307f9e0bdd74576bfd1c157610ad9e9" modify_external_catalogs: true
W20260810 07:52:46.695298 792 leader_election.cc:343] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.696583 1071 tablet_bootstrap.cc:492] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:46.697064 1071 ts_tablet_manager.cc:1403] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.304s user 0.005s sys 0.010s
I20260810 07:52:46.698029 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_ff19276d47ee41718e958f3f15343ac9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:46.699682 1109 raft_consensus.cc:493] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.699887 1071 raft_consensus.cc:359] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.700172 1109 raft_consensus.cc:515] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:46.701417 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff19276d47ee41718e958f3f15343ac9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.702666 1081 raft_consensus.cc:515] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:46.704900 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_452dd2adccb043c685e5eb8af81bd4db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:46.706709 309 leader_election.cc:343] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: f176f0909d064a93ab2384ac037123df
I20260810 07:52:46.707603 1081 leader_election.cc:290] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.680455 1064 consensus_queue.cc:260] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:46.717631 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_6454dbcc43ab45ff8313cbf3e725d310" } new_table_name: "default.Table_faede62cb1c04a3ebc3883318c37608d" modify_external_catalogs: true
I20260810 07:52:46.719959 1149 raft_consensus.cc:493] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.720683 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f176f0909d064a93ab2384ac037123df" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:46.721343 678 raft_consensus.cc:3060] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.721937 1064 ts_tablet_manager.cc:1434] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.045s user 0.008s sys 0.000s
I20260810 07:52:46.721866 380 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:46.723248 1064 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.729172 1065 ts_tablet_manager.cc:1916] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:46.737920 1150 tablet.cc:1724] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:46.745711 1064 tablet_bootstrap.cc:654] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.720449 1149 raft_consensus.cc:515] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.757129 32470 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_612e3c4860664a5a964122487ce93bed" } modify_external_catalogs: true
I20260810 07:52:46.757685 32470 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_612e3c4860664a5a964122487ce93bed" } modify_external_catalogs: true
W20260810 07:52:46.761016 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6f37bba532b4d8bbc00e6d91f4c1820 consensus state has no leader
I20260810 07:52:46.774367 1149 leader_election.cc:290] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.775178 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f37bba532b4d8bbc00e6d91f4c1820" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.775520 1142 leader_election.cc:290] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.775877 677 raft_consensus.cc:2468] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:46.776005 1142 raft_consensus.cc:493] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:52:46.772786 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08520a2ac0f64c3abd534a32100839a4 has no consensus state
I20260810 07:52:46.776422 1142 raft_consensus.cc:515] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:46.785799 1142 leader_election.cc:290] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.792295 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.777736 792 leader_election.cc:304] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:52:46.795305 792 leader_election.cc:343] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.795727 792 leader_election.cc:304] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:46.796463 1142 raft_consensus.cc:2804] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:46.796840 1142 raft_consensus.cc:493] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:46.797179 1142 raft_consensus.cc:3060] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.802862 1064 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:46.803288 1064 ts_tablet_manager.cc:1403] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.080s user 0.015s sys 0.005s
I20260810 07:52:46.804579 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:46.806712 1142 raft_consensus.cc:515] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.810297 1153 raft_consensus.cc:2749] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:46.811350 1142 leader_election.cc:290] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.812005 1065 ts_tablet_manager.cc:1929] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:46.812382 1065 log.cc:1199] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/0075243274794151ad4e2e0130ae04ef
I20260810 07:52:46.813409 1065 ts_tablet_manager.cc:1950] T 0075243274794151ad4e2e0130ae04ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:46.815964 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_a4b3c1181096402598c18155b4383ecb [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]) successfully deleted
W20260810 07:52:46.819829 792 leader_election.cc:343] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.819885 1064 raft_consensus.cc:359] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.820705 1064 raft_consensus.cc:385] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.819168 32469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_52cc17edf52f44798e287e045fe08934" } modify_external_catalogs: true
I20260810 07:52:46.821112 1064 raft_consensus.cc:740] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.821537 32469 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_52cc17edf52f44798e287e045fe08934" } modify_external_catalogs: true
I20260810 07:52:46.839605 1064 consensus_queue.cc:260] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.845764 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a0d413eac6aa41fb8b6eddaf6f051c72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.848528 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0d413eac6aa41fb8b6eddaf6f051c72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.774008 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f37bba532b4d8bbc00e6d91f4c1820" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.828501 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f37bba532b4d8bbc00e6d91f4c1820" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:46.700562 1071 raft_consensus.cc:385] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.828217 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:46.861029 1071 raft_consensus.cc:740] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:52:46.864666 792 leader_election.cc:343] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:46.865134 792 leader_election.cc:304] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:46.866271 1142 raft_consensus.cc:2749] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:46.861811 1071 consensus_queue.cc:260] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.871759 1071 ts_tablet_manager.cc:1434] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.174s user 0.006s sys 0.009s
I20260810 07:52:46.883126 1071 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:46.893481 415 raft_consensus.cc:2468] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:46.896126 1071 tablet_bootstrap.cc:654] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:46.817488 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f37bba532b4d8bbc00e6d91f4c1820" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:46.898908 677 raft_consensus.cc:3060] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:46.901799 1146 tablet.cc:1724] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:46.906178 1109 leader_election.cc:290] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:46.914943 1142 raft_consensus.cc:493] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:46.915123 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:46.915454 1142 raft_consensus.cc:515] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.915925 896 raft_consensus.cc:2468] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:46.865163 1064 ts_tablet_manager.cc:1434] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.062s user 0.009s sys 0.000s
I20260810 07:52:46.917078 1064 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:46.917367 1142 leader_election.cc:290] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:46.919510 1047 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:46.919996 1047 ts_tablet_manager.cc:1403] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.443s user 0.011s sys 0.006s
I20260810 07:52:46.921773 574 leader_election.cc:304] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:46.922396 1047 raft_consensus.cc:359] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:46.923202 1047 raft_consensus.cc:385] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:46.923547 1047 raft_consensus.cc:740] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:46.938483 1109 raft_consensus.cc:2804] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:46.938885 1109 raft_consensus.cc:493] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:46.940903 1109 raft_consensus.cc:3060] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:46.949540 1150 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.210s user 0.000s sys 0.003s
W20260810 07:52:46.948518 792 leader_election.cc:343] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: 5b76191c5bed41509344376f1810e7aa
W20260810 07:52:46.967927 792 leader_election.cc:343] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 5b76191c5bed41509344376f1810e7aa
I20260810 07:52:46.977509 32468 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]). Expected version 3 got 0
I20260810 07:52:46.982013 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_7e88718860ff4246b3dcf39078c78463" } new_table_name: "default.Table_7423fb2519b540d6a3f8364376cd7148" modify_external_catalogs: true
I20260810 07:52:46.983026 872 tablet_service.cc:1511] Processing CreateTablet for tablet e44b480f5f3d4f239eae2586c1d2001a (DEFAULT_TABLE table=default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:46.984210 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44b480f5f3d4f239eae2586c1d2001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:46.983996 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_392a81b62c08432fa1f4a7747cf566f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.985695 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_392a81b62c08432fa1f4a7747cf566f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:46.991189 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c697e7b046864325af96fafa45e60fb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:46.992909 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c697e7b046864325af96fafa45e60fb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:46.996003 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 252859us
I20260810 07:52:46.968343 792 leader_election.cc:304] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:47.003921 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_792d89d794dd460e8a64665f11bd3641" } new_table_name: "default.Table_027a339e6726492e8900a305166515ec" modify_external_catalogs: true
I20260810 07:52:47.006321 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:47.010670 573 leader_election.cc:343] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.013059 380 tablet_service.cc:1511] Processing CreateTablet for tablet 08520a2ac0f64c3abd534a32100839a4 (DEFAULT_TABLE table=default.Table_c307f9e0bdd74576bfd1c157610ad9e9 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.014299 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08520a2ac0f64c3abd534a32100839a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.015156 32469 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 45ff852e62624751b8d26fdf1f9b2ff7
I20260810 07:52:47.021695 639 tablet_service.cc:1511] Processing CreateTablet for tablet 08520a2ac0f64c3abd534a32100839a4 (DEFAULT_TABLE table=default.Table_c307f9e0bdd74576bfd1c157610ad9e9 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.010308 869 tablet_service.cc:1511] Processing CreateTablet for tablet 08520a2ac0f64c3abd534a32100839a4 (DEFAULT_TABLE table=default.Table_c307f9e0bdd74576bfd1c157610ad9e9 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.023000 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08520a2ac0f64c3abd534a32100839a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.023010 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08520a2ac0f64c3abd534a32100839a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.024530 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3a887c70a4ef45ccace031aa6fa907de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.026258 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a887c70a4ef45ccace031aa6fa907de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.027709 871 tablet_service.cc:1511] Processing CreateTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d (DEFAULT_TABLE table=default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.028865 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f09a245fe4c1ea4e391166eaf3d3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:47.030993 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 63012.7us
I20260810 07:52:47.033635 1075 raft_consensus.cc:2749] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:47.039510 677 raft_consensus.cc:2468] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:47.044052 1103 ts_tablet_manager.cc:1929] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.044505 1103 log.cc:1199] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:47.046093 792 leader_election.cc:304] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:47.047062 1064 tablet_bootstrap.cc:654] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.052194 1075 raft_consensus.cc:2804] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:47.052773 1075 raft_consensus.cc:697] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:47.053913 1142 raft_consensus.cc:493] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.054352 1142 raft_consensus.cc:515] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.056268 1142 leader_election.cc:290] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:47.058179 574 consensus_peers.cc:597] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0075243274794151ad4e2e0130ae04ef. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:47.059155 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99d1bfe95e5d4d8697584b05b28339f3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:47.065845 642 tablet_service.cc:1511] Processing CreateTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d (DEFAULT_TABLE table=default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.067083 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f09a245fe4c1ea4e391166eaf3d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.075580 869 tablet_service.cc:1511] Processing CreateTablet for tablet d397310bcfd2470e95fc4b0654749aa4 (DEFAULT_TABLE table=default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.076879 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d397310bcfd2470e95fc4b0654749aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.055263 1075 consensus_queue.cc:237] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.089501 1109 raft_consensus.cc:515] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.092350 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:47.093031 896 raft_consensus.cc:3060] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:47.058091 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99d1bfe95e5d4d8697584b05b28339f3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:47.062295 384 tablet_service.cc:1511] Processing CreateTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d (DEFAULT_TABLE table=default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.097601 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f09a245fe4c1ea4e391166eaf3d3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:47.098028 792 leader_election.cc:343] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.102377 896 raft_consensus.cc:2468] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:47.114056 678 raft_consensus.cc:2468] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:52:47.115518 308 leader_election.cc:304] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:47.115722 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 1 got 0
I20260810 07:52:47.116859 1148 raft_consensus.cc:2804] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:47.117048 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]). Expected version 3 got 0
I20260810 07:52:47.117393 1148 raft_consensus.cc:697] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:52:47.118309 1148 consensus_queue.cc:237] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.121174 32465 catalog_manager.cc:5719] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:47.122911 574 leader_election.cc:304] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:47.124205 1154 raft_consensus.cc:2804] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:47.126276 1154 raft_consensus.cc:697] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:47.129906 1154 consensus_queue.cc:237] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.098129 1103 ts_tablet_manager.cc:1950] T 0f46f1ca4dbe4d2c8ead06651437e042 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:47.136526 1103 tablet_replica.cc:333] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:46.924386 1047 consensus_queue.cc:260] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.137429 676 raft_consensus.cc:2468] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:47.140908 792 leader_election.cc:304] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:47.142163 1142 raft_consensus.cc:2804] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:47.142527 1142 raft_consensus.cc:493] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:47.142936 1142 raft_consensus.cc:3060] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:47.136222 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 0f46f1ca4dbe4d2c8ead06651437e042 (table default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]) successfully deleted
W20260810 07:52:47.144294 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet e13d8a5248d54b00aa4a47874561745e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e13d8a5248d54b00aa4a47874561745e already in progress: creating tablet
I20260810 07:52:47.146409 1064 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:47.146955 1064 ts_tablet_manager.cc:1403] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.230s user 0.011s sys 0.000s
I20260810 07:52:47.149765 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_1992a5b4d5144b85986cec15a6b24446 [id=2e56b185007f4d798dba6e17d0b14818]). Expected version 1 got 0
I20260810 07:52:47.151190 32470 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:47.154592 1142 raft_consensus.cc:515] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.154978 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:47.155761 1065 tablet_replica.cc:333] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:47.156556 1065 raft_consensus.cc:2243] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.157198 1065 raft_consensus.cc:2272] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.157835 1142 leader_election.cc:290] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:47.167764 869 tablet_service.cc:1511] Processing CreateTablet for tablet f353e1e07485400a964f8e05b0691cc6 (DEFAULT_TABLE table=default.Table_792d89d794dd460e8a64665f11bd3641 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.168936 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f353e1e07485400a964f8e05b0691cc6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:47.171742 32462 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7a1eb27f536648c888a5acc3e940e4c0 has no consensus state
I20260810 07:52:47.158560 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99d1bfe95e5d4d8697584b05b28339f3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:47.184707 1103 raft_consensus.cc:2243] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:52:47.185261 792 leader_election.cc:343] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.189349 1103 raft_consensus.cc:2272] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.192408 1103 ts_tablet_manager.cc:1916] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.200584 1065 ts_tablet_manager.cc:1916] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.209826 1064 raft_consensus.cc:359] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.210533 1064 raft_consensus.cc:385] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.210805 1064 raft_consensus.cc:740] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.216856 1064 consensus_queue.cc:260] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.221303 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b21be299a7664348b1a5fb51d9f031ed" } modify_external_catalogs: true
I20260810 07:52:47.221964 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b21be299a7664348b1a5fb51d9f031ed" } modify_external_catalogs: true
W20260810 07:52:47.223026 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7a1eb27f536648c888a5acc3e940e4c0 has no consensus state
I20260810 07:52:47.227026 1064 ts_tablet_manager.cc:1434] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.080s user 0.007s sys 0.000s
I20260810 07:52:47.159653 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:47.164491 377 tablet_service.cc:1511] Processing CreateTablet for tablet d397310bcfd2470e95fc4b0654749aa4 (DEFAULT_TABLE table=default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.231384 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d397310bcfd2470e95fc4b0654749aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.234117 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:47.234716 646 tablet_service.cc:1511] Processing CreateTablet for tablet d397310bcfd2470e95fc4b0654749aa4 (DEFAULT_TABLE table=default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.235508 1109 leader_election.cc:290] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:47.235846 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d397310bcfd2470e95fc4b0654749aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.236680 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:47.159329 382 tablet_service.cc:1511] Processing CreateTablet for tablet e44b480f5f3d4f239eae2586c1d2001a (DEFAULT_TABLE table=default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.238214 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44b480f5f3d4f239eae2586c1d2001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.196381 376 tablet_service.cc:1511] Processing CreateTablet for tablet f353e1e07485400a964f8e05b0691cc6 (DEFAULT_TABLE table=default.Table_792d89d794dd460e8a64665f11bd3641 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.243930 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f353e1e07485400a964f8e05b0691cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.245116 1064 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:47.257889 1065 ts_tablet_manager.cc:1929] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.258376 1065 log.cc:1199] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:47.259064 1065 ts_tablet_manager.cc:1950] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
W20260810 07:52:47.260969 573 leader_election.cc:343] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.236819 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99d1bfe95e5d4d8697584b05b28339f3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:47.263264 678 raft_consensus.cc:3060] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:47.229986 1071 tablet_bootstrap.cc:492] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:47.265046 1071 ts_tablet_manager.cc:1403] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.382s user 0.010s sys 0.001s
I20260810 07:52:47.280134 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_72b2a49c2b424ed08ade35de3caa325b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:47.281661 638 tablet_service.cc:1511] Processing CreateTablet for tablet f353e1e07485400a964f8e05b0691cc6 (DEFAULT_TABLE table=default.Table_792d89d794dd460e8a64665f11bd3641 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:47.282025 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72b2a49c2b424ed08ade35de3caa325b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.283758 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f353e1e07485400a964f8e05b0691cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.287139 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 99d1bfe95e5d4d8697584b05b28339f3 (table default.Table_612e3c4860664a5a964122487ce93bed [id=0d62e87e4971408faa84ce2850fe01e3]) successfully deleted
I20260810 07:52:47.267763 1071 raft_consensus.cc:359] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.288335 1071 raft_consensus.cc:385] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.297014 1071 raft_consensus.cc:740] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.297734 1068 ts_tablet_manager.cc:1929] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:47.298825 1068 log.cc:1199] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/0f46f1ca4dbe4d2c8ead06651437e042
I20260810 07:52:47.298681 1071 consensus_queue.cc:260] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.301230 644 tablet_service.cc:1511] Processing CreateTablet for tablet e44b480f5f3d4f239eae2586c1d2001a (DEFAULT_TABLE table=default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.302605 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44b480f5f3d4f239eae2586c1d2001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.306154 1071 ts_tablet_manager.cc:1434] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.041s user 0.004s sys 0.006s
I20260810 07:52:47.307772 1071 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:47.309139 1047 ts_tablet_manager.cc:1434] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.389s user 0.008s sys 0.000s
I20260810 07:52:47.310537 1047 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:47.320515 678 raft_consensus.cc:2468] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
W20260810 07:52:47.345072 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f353e1e07485400a964f8e05b0691cc6 consensus state has no leader
I20260810 07:52:47.346630 1071 tablet_bootstrap.cc:654] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.347995 792 leader_election.cc:304] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
W20260810 07:52:47.348641 792 raft_consensus.cc:2686] T 99d1bfe95e5d4d8697584b05b28339f3 P 9e545e0597e24d61b7265abed2ccdcfb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260810 07:52:47.300405 1068 ts_tablet_manager.cc:1950] T 0f46f1ca4dbe4d2c8ead06651437e042 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:47.355348 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 0f46f1ca4dbe4d2c8ead06651437e042 (table default.Table_f2c5c63a66cb40bf81ebe3a0f890e3d4 [id=8e47520b240b41288a7d8ca56c9af395]) successfully deleted
I20260810 07:52:47.358793 1068 tablet_replica.cc:333] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:47.382805 1064 tablet_bootstrap.cc:654] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.386049 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]). Expected version 3 got 0
I20260810 07:52:47.386101 1146 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 16) took 1055 ms. Trace:
I20260810 07:52:47.335747 1047 tablet_bootstrap.cc:654] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.392133 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet e44b480f5f3d4f239eae2586c1d2001a
I20260810 07:52:47.395855 642 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:47.395946 377 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:47.401561 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:47.404260 869 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:47.410202 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 99d1bfe95e5d4d8697584b05b28339f3 because the tablet was not found. No further retry: Not found: Tablet not found: 99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:47.415202 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 0075243274794151ad4e2e0130ae04ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:47.430209 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet e44b480f5f3d4f239eae2586c1d2001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e44b480f5f3d4f239eae2586c1d2001a already in progress: creating tablet
I20260810 07:52:47.438971 872 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:47.440062 1065 tablet_replica.cc:333] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:47.441294 1065 raft_consensus.cc:2243] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.441987 1065 raft_consensus.cc:2272] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.443858 1065 ts_tablet_manager.cc:1916] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:47.449963 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 0075243274794151ad4e2e0130ae04ef because the tablet was not found. No further retry: Not found: Tablet not found: 0075243274794151ad4e2e0130ae04ef
I20260810 07:52:47.386720 1146 rpcz_store.cc:276] 0810 07:52:46.330388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.330585 (+ 197us) service_pool.cc:224] Handling call
0810 07:52:47.386026 (+1055441us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:46.356656 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:52:46.357138 (+ 482us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:52:46.357178 (+ 40us) alter_schema_op.cc:66] Acquired schema lock
0810 07:52:46.357302 (+ 124us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:52:46.357480 (+ 178us) alter_schema_op.cc:128] START. Timestamp: 1786348366357389
0810 07:52:46.357514 (+ 34us) op_driver.cc:348] REPLICATION: starting
0810 07:52:46.358220 (+ 706us) log.cc:844] Serialized 173 byte log entry
0810 07:52:46.691821 (+333601us) op_driver.cc:464] REPLICATION: finished
0810 07:52:46.901506 (+209685us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:52:46.903487 (+ 1981us) log.cc:844] Serialized 5 byte log entry
0810 07:52:47.338384 (+434897us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.376392 (+ 38008us) log.cc:844] Serialized 14 byte log entry
0810 07:52:47.385807 (+ 9415us) alter_schema_op.cc:73] Released schema lock
0810 07:52:47.385851 (+ 44us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":209404,"mutex_wait_us":1423,"prepare.queue_time_us":23895,"prepare.run_cpu_time_us":2227,"prepare.run_wall_time_us":72027,"raft.queue_time_us":1674,"raft.run_cpu_time_us":1341,"raft.run_wall_time_us":1340,"replication_time_us":334118,"thread_start_us":238570,"threads_started":2}]]}
I20260810 07:52:47.461267 1119 mvcc.cc:204] Tried to move back new op lower bound from 7316882908599865344 to 7316882906158092288. Current Snapshot: MvccSnapshot[applied={T|T < 7316882908599865344}]
I20260810 07:52:47.461659 1068 raft_consensus.cc:2243] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.462487 1068 raft_consensus.cc:2272] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.465289 1068 ts_tablet_manager.cc:1916] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.468983 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_027a339e6726492e8900a305166515ec" } new_table_name: "default.Table_0f6aa0ec81114d78abb2ad53a02bf930" modify_external_catalogs: true
I20260810 07:52:47.477820 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_2101337b99f744bebd7fe77c3777e236"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.480067 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2101337b99f744bebd7fe77c3777e236 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.475113 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_937b2a00ea624497b31e38f446787d9f" } modify_external_catalogs: true
I20260810 07:52:47.481190 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_937b2a00ea624497b31e38f446787d9f" } modify_external_catalogs: true
W20260810 07:52:47.493353 573 consensus_peers.cc:597] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:47.499168 1065 ts_tablet_manager.cc:1929] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.509939 1065 log.cc:1199] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/e13d8a5248d54b00aa4a47874561745e
I20260810 07:52:47.510871 1065 ts_tablet_manager.cc:1950] T e13d8a5248d54b00aa4a47874561745e P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:47.514217 1064 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:47.514652 1064 ts_tablet_manager.cc:1403] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.270s user 0.012s sys 0.009s
I20260810 07:52:47.524456 1103 ts_tablet_manager.cc:1929] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:47.525452 1103 log.cc:1199] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/0075243274794151ad4e2e0130ae04ef
I20260810 07:52:47.526674 1103 ts_tablet_manager.cc:1950] T 0075243274794151ad4e2e0130ae04ef P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:47.530625 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_a4b3c1181096402598c18155b4383ecb [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]) successfully deleted
I20260810 07:52:47.525571 1071 tablet_bootstrap.cc:492] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:47.531806 1071 ts_tablet_manager.cc:1403] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.224s user 0.026s sys 0.029s
I20260810 07:52:47.533401 1064 raft_consensus.cc:359] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.534102 1064 raft_consensus.cc:385] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.534365 1064 raft_consensus.cc:740] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.534049 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_51a621150c8343f4a0cc9e0e8bb61580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.536150 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51a621150c8343f4a0cc9e0e8bb61580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.536092 1071 raft_consensus.cc:359] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.537055 1071 raft_consensus.cc:385] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.537359 1071 raft_consensus.cc:740] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.538566 1103 tablet_replica.cc:333] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:47.539525 1103 raft_consensus.cc:2243] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:47.540619 1103 pending_rounds.cc:70] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Trying to abort 1 pending ops.
I20260810 07:52:47.541071 1103 pending_rounds.cc:77] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882911729827840 op_type: NO_OP noop_request { }
I20260810 07:52:47.542707 1103 raft_consensus.cc:2889] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:47.543257 1103 raft_consensus.cc:2272] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.543237 1068 ts_tablet_manager.cc:1929] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.543637 1068 log.cc:1199] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/f176f0909d064a93ab2384ac037123df
I20260810 07:52:47.541884 1071 consensus_queue.cc:260] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.544512 1064 consensus_queue.cc:260] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.555106 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet e13d8a5248d54b00aa4a47874561745e (table default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]) successfully deleted
I20260810 07:52:47.556636 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_480a23c1ffaa4896a093c62ceff5822e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.558602 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_480a23c1ffaa4896a093c62ceff5822e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.562582 1064 ts_tablet_manager.cc:1434] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260810 07:52:47.565842 1068 ts_tablet_manager.cc:1950] T f176f0909d064a93ab2384ac037123df P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:47.571542 1071 ts_tablet_manager.cc:1434] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.039s user 0.005s sys 0.003s
I20260810 07:52:47.573454 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c697e7b046864325af96fafa45e60fb6" } new_table_name: "default.Table_94500f9e785c470e8b4ee8981ee5d379" modify_external_catalogs: true
I20260810 07:52:47.572629 1047 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:47.577207 1047 ts_tablet_manager.cc:1403] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.267s user 0.013s sys 0.012s
W20260810 07:52:47.578750 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260810 07:52:47.579819 1047 raft_consensus.cc:359] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.581040 1075 raft_consensus.cc:493] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.581652 1075 raft_consensus.cc:515] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.583787 1075 leader_election.cc:290] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:47.585235 1047 raft_consensus.cc:385] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.585862 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:47.586118 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:47.586621 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f353e1e07485400a964f8e05b0691cc6 consensus state has no leader
I20260810 07:52:47.585561 1047 raft_consensus.cc:740] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:52:47.587652 792 leader_election.cc:343] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.587551 1047 consensus_queue.cc:260] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:47.588340 792 leader_election.cc:343] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.588696 792 leader_election.cc:304] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:47.589803 1075 raft_consensus.cc:2749] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:47.594225 1071 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:47.572779 1103 ts_tablet_manager.cc:1916] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.608700 1068 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 20) took 1077 ms. Trace:
I20260810 07:52:47.609246 1068 rpcz_store.cc:276] 0810 07:52:46.531150 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.535436 (+ 4286us) service_pool.cc:224] Handling call
0810 07:52:47.357328 (+821892us) ts_tablet_manager.cc:1151] Deleting tablet f176f0909d064a93ab2384ac037123df
0810 07:52:47.357674 (+ 346us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:47.499034 (+141360us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.543091 (+ 44057us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.608630 (+ 65539us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94963,"tablet-delete.queue_time_us":815976}
I20260810 07:52:47.576661 1064 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:47.610996 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet f176f0909d064a93ab2384ac037123df (table default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]) successfully deleted
I20260810 07:52:47.612418 1068 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 21) took 1059 ms. Trace:
I20260810 07:52:47.612864 1068 rpcz_store.cc:276] 0810 07:52:46.553060 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.553643 (+ 583us) service_pool.cc:224] Handling call
0810 07:52:47.612010 (+1058367us) ts_tablet_manager.cc:1151] Deleting tablet e13d8a5248d54b00aa4a47874561745e
0810 07:52:47.612060 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:47.612356 (+ 296us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1057718}
W20260810 07:52:47.614411 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet e13d8a5248d54b00aa4a47874561745e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e13d8a5248d54b00aa4a47874561745e already in progress: creating tablet
I20260810 07:52:47.615422 1068 tablet_replica.cc:333] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:47.616271 1068 raft_consensus.cc:2243] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:47.617556 1068 pending_rounds.cc:70] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:47.617851 1068 pending_rounds.cc:77] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7316882912841928704 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "0075243274794151ad4e2e0130ae04ef" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_a4b3c1181096402598c18155b4383ecb" dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" new_extra_config { } }
W20260810 07:52:47.618861 1068 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57706: Aborted: Op aborted
I20260810 07:52:47.619454 1068 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 17) took 1100 ms. Trace:
I20260810 07:52:47.619805 1068 rpcz_store.cc:276] 0810 07:52:46.530112 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.530236 (+ 124us) service_pool.cc:224] Handling call
0810 07:52:47.619399 (+1089163us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:46.537321 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:52:46.537963 (+ 642us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:52:47.392888 (+854925us) alter_schema_op.cc:66] Acquired schema lock
0810 07:52:47.392976 (+ 88us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:52:47.393124 (+ 148us) alter_schema_op.cc:128] START. Timestamp: 1786348367393049
0810 07:52:47.393147 (+ 23us) op_driver.cc:348] REPLICATION: starting
0810 07:52:47.393718 (+ 571us) log.cc:844] Serialized 173 byte log entry
0810 07:52:47.618696 (+224978us) op_driver.cc:464] REPLICATION: finished
0810 07:52:47.618758 (+ 62us) op_driver.cc:389] HandleFailure(Aborted: Op aborted)
0810 07:52:47.618789 (+ 31us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":1420,"prepare.run_cpu_time_us":33226,"prepare.run_wall_time_us":861885,"raft.queue_time_us":1600,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":753,"replication_time_us":225430}]]}
I20260810 07:52:47.624114 1068 raft_consensus.cc:2272] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.626660 1068 ts_tablet_manager.cc:1916] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.629834 1064 tablet_bootstrap.cc:654] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.634202 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_72b2a49c2b424ed08ade35de3caa325b" } modify_external_catalogs: true
I20260810 07:52:47.634812 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_72b2a49c2b424ed08ade35de3caa325b" } modify_external_catalogs: true
I20260810 07:52:47.608922 1047 ts_tablet_manager.cc:1434] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.031s user 0.005s sys 0.002s
I20260810 07:52:47.639148 1047 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:47.645138 1071 tablet_bootstrap.cc:654] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.654168 1068 ts_tablet_manager.cc:1929] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:47.654744 1068 log.cc:1199] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/0075243274794151ad4e2e0130ae04ef
I20260810 07:52:47.655736 1068 ts_tablet_manager.cc:1950] T 0075243274794151ad4e2e0130ae04ef P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:47.658351 1068 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 23) took 1053 ms. Trace:
I20260810 07:52:47.658769 1068 rpcz_store.cc:276] 0810 07:52:46.605002 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.605156 (+ 154us) service_pool.cc:224] Handling call
0810 07:52:47.615263 (+1010107us) ts_tablet_manager.cc:1151] Deleting tablet 0075243274794151ad4e2e0130ae04ef
0810 07:52:47.615308 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:47.635147 (+ 19839us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.654030 (+ 18883us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.658278 (+ 4248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1009448}
I20260810 07:52:47.660552 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 0075243274794151ad4e2e0130ae04ef (table default.Table_a4b3c1181096402598c18155b4383ecb [id=938b0d25a46e4a75bcdbcc37f3bc6f2c]) successfully deleted
I20260810 07:52:47.661722 1068 tablet_replica.cc:333] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:47.662891 1068 raft_consensus.cc:2243] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.663977 1075 raft_consensus.cc:493] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.664469 1075 raft_consensus.cc:515] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.666625 1075 leader_election.cc:290] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:47.669480 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:47.670815 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:47.668041 1047 tablet_bootstrap.cc:654] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:47.672884 792 leader_election.cc:343] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:47.673653 792 leader_election.cc:343] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.674108 792 leader_election.cc:304] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:47.675341 1075 raft_consensus.cc:2749] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:47.663455 1068 raft_consensus.cc:2272] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.677416 1064 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:47.677843 1064 ts_tablet_manager.cc:1403] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.101s user 0.009s sys 0.000s
I20260810 07:52:47.678153 1068 ts_tablet_manager.cc:1916] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.680320 1064 raft_consensus.cc:359] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.680984 1064 raft_consensus.cc:385] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.681252 1064 raft_consensus.cc:740] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.690604 1047 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:47.691076 1047 ts_tablet_manager.cc:1403] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.002s
I20260810 07:52:47.693833 1047 raft_consensus.cc:359] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.694546 1047 raft_consensus.cc:385] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.694844 1047 raft_consensus.cc:740] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.696202 1068 ts_tablet_manager.cc:1929] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.696694 1068 log.cc:1199] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:47.697536 1068 ts_tablet_manager.cc:1950] T 99d1bfe95e5d4d8697584b05b28339f3 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:47.697999 1047 consensus_queue.cc:260] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:47.699074 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet acf740de7abb40bfabfc3a7859cc159b has no consensus state
I20260810 07:52:47.700636 1064 consensus_queue.cc:260] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.710160 1047 ts_tablet_manager.cc:1434] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.019s user 0.006s sys 0.002s
I20260810 07:52:47.716357 1103 ts_tablet_manager.cc:1929] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:47.711094 1047 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:47.721822 1075 consensus_queue.cc:1048] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:47.726336 1064 ts_tablet_manager.cc:1434] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.048s user 0.011s sys 0.004s
I20260810 07:52:47.727301 1064 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:47.729841 32464 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 2b82775a13fb4b96aaf06f2bb8c11fb2
I20260810 07:52:47.724596 1047 tablet_bootstrap.cc:654] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.745417 1064 tablet_bootstrap.cc:654] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.745468 1103 log.cc:1199] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/f176f0909d064a93ab2384ac037123df
I20260810 07:52:47.718618 1071 tablet_bootstrap.cc:492] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:47.746071 1071 ts_tablet_manager.cc:1403] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.152s user 0.004s sys 0.009s
I20260810 07:52:47.748642 1071 raft_consensus.cc:359] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.750239 1103 ts_tablet_manager.cc:1950] T f176f0909d064a93ab2384ac037123df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:47.752287 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 99d1bfe95e5d4d8697584b05b28339f3 (table default.Table_612e3c4860664a5a964122487ce93bed [id=0d62e87e4971408faa84ce2850fe01e3]) successfully deleted
I20260810 07:52:47.752792 1103 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 21) took 1031 ms. Trace:
W20260810 07:52:47.754933 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet e44b480f5f3d4f239eae2586c1d2001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e44b480f5f3d4f239eae2586c1d2001a already in progress: creating tablet
I20260810 07:52:47.759794 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f571ac3561af4ca9b497338a81e1f9a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.761670 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f571ac3561af4ca9b497338a81e1f9a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.773847 872 tablet_service.cc:1511] Processing CreateTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.774209 1064 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:47.774627 1064 ts_tablet_manager.cc:1403] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.000s
I20260810 07:52:47.774006 32465 catalog_manager.cc:5719] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:47.775048 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1373ebbd3ba4262806bfc17823d25b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.775477 32465 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 1 got 0
I20260810 07:52:47.777125 1064 raft_consensus.cc:359] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.777783 1064 raft_consensus.cc:385] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.778034 1064 raft_consensus.cc:740] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.778895 1064 consensus_queue.cc:260] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.778404 32460 catalog_manager.cc:5719] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:47.780575 644 tablet_service.cc:1511] Processing CreateTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.781430 869 tablet_service.cc:1511] Processing CreateTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 (DEFAULT_TABLE table=default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.781831 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1373ebbd3ba4262806bfc17823d25b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.782411 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fcae5b2e5448bf89b1c0587f46ad40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.785207 638 tablet_service.cc:1511] Processing CreateTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 (DEFAULT_TABLE table=default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.786253 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fcae5b2e5448bf89b1c0587f46ad40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.789481 646 tablet_service.cc:1511] Processing CreateTablet for tablet 69c05d6e49204eedbb305f66dfa2316c (DEFAULT_TABLE table=default.Table_a0d413eac6aa41fb8b6eddaf6f051c72 [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.790548 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c05d6e49204eedbb305f66dfa2316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.779844 32460 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]). Expected version 3 got 0
I20260810 07:52:47.792783 32460 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_1992a5b4d5144b85986cec15a6b24446 [id=2e56b185007f4d798dba6e17d0b14818]). Expected version 1 got 0
I20260810 07:52:47.793529 32460 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:47.794090 871 tablet_service.cc:1511] Processing CreateTablet for tablet 69c05d6e49204eedbb305f66dfa2316c (DEFAULT_TABLE table=default.Table_a0d413eac6aa41fb8b6eddaf6f051c72 [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.794268 32460 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:47.795207 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c05d6e49204eedbb305f66dfa2316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.749404 1071 raft_consensus.cc:385] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.782390 377 tablet_service.cc:1511] Processing CreateTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.800601 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1373ebbd3ba4262806bfc17823d25b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.783842 376 tablet_service.cc:1511] Processing CreateTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 (DEFAULT_TABLE table=default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.804788 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fcae5b2e5448bf89b1c0587f46ad40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.785161 382 tablet_service.cc:1511] Processing CreateTablet for tablet 69c05d6e49204eedbb305f66dfa2316c (DEFAULT_TABLE table=default.Table_a0d413eac6aa41fb8b6eddaf6f051c72 [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.809654 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c05d6e49204eedbb305f66dfa2316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.793870 1064 ts_tablet_manager.cc:1434] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260810 07:52:47.796653 415 raft_consensus.cc:3060] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:47.813097 380 tablet_service.cc:1511] Processing CreateTablet for tablet c9920a33cc93419ca3284a0b009236df (DEFAULT_TABLE table=default.Table_ff19276d47ee41718e958f3f15343ac9 [id=6d38995588ed44479ad04afe299ed43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.814218 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9920a33cc93419ca3284a0b009236df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.827920 1064 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:47.834096 1064 tablet_bootstrap.cc:654] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.834785 1047 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:47.835207 1047 ts_tablet_manager.cc:1403] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.124s user 0.010s sys 0.002s
I20260810 07:52:47.840822 1071 raft_consensus.cc:740] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.841857 1071 consensus_queue.cc:260] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.844839 642 tablet_service.cc:1511] Processing CreateTablet for tablet c9920a33cc93419ca3284a0b009236df (DEFAULT_TABLE table=default.Table_ff19276d47ee41718e958f3f15343ac9 [id=6d38995588ed44479ad04afe299ed43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.846000 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9920a33cc93419ca3284a0b009236df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.848178 1064 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:47.848623 1064 ts_tablet_manager.cc:1403] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.000s
I20260810 07:52:47.850929 1071 ts_tablet_manager.cc:1434] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.105s user 0.003s sys 0.002s
I20260810 07:52:47.851301 1064 raft_consensus.cc:359] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.852008 1064 raft_consensus.cc:385] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.852312 1064 raft_consensus.cc:740] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.856074 1064 consensus_queue.cc:260] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.861289 868 tablet_service.cc:1511] Processing CreateTablet for tablet c9920a33cc93419ca3284a0b009236df (DEFAULT_TABLE table=default.Table_ff19276d47ee41718e958f3f15343ac9 [id=6d38995588ed44479ad04afe299ed43b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.862491 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9920a33cc93419ca3284a0b009236df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.865790 1047 raft_consensus.cc:359] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:47.866513 1047 raft_consensus.cc:385] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:47.866793 1047 raft_consensus.cc:740] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:47.867630 1047 consensus_queue.cc:260] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:47.869014 1075 consensus_queue.cc:1048] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:47.868844 1103 rpcz_store.cc:276] 0810 07:52:46.721476 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:46.721680 (+ 204us) service_pool.cc:224] Handling call
0810 07:52:47.538343 (+816663us) ts_tablet_manager.cc:1151] Deleting tablet f176f0909d064a93ab2384ac037123df
0810 07:52:47.538411 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:47.638279 (+ 99868us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.716230 (+ 77951us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:47.752681 (+ 36451us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":816001}
I20260810 07:52:47.870911 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet f176f0909d064a93ab2384ac037123df (table default.Table_efcdf13d05104f71aa1f595ee2834d3c [id=d32f2446b5284a9d9e367b26e78ca68c]) successfully deleted
I20260810 07:52:47.871094 1148 raft_consensus.cc:493] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.871673 1148 raft_consensus.cc:515] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.873706 1075 raft_consensus.cc:493] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.873919 385 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1eb27f536648c888a5acc3e940e4c0 (DEFAULT_TABLE table=default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.874308 1148 leader_election.cc:290] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:47.874222 1075 raft_consensus.cc:515] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.875198 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1eb27f536648c888a5acc3e940e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.877701 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:47.878562 896 raft_consensus.cc:2393] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:47.882395 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:47.886466 792 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.876394 1075 leader_election.cc:290] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:47.889807 1064 ts_tablet_manager.cc:1434] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260810 07:52:47.894297 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a0d413eac6aa41fb8b6eddaf6f051c72" } new_table_name: "default.Table_5eac9bcc1944426bac3bdacfcb745f39" modify_external_catalogs: true
I20260810 07:52:47.898159 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f3fa658df8764c2aa73aa960d170e2a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:47.900142 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3fa658df8764c2aa73aa960d170e2a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.901678 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0741e2ea30224c6180516e0791c77331"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:47.902259 1064 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:47.903563 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0741e2ea30224c6180516e0791c77331 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.905263 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_58e123d14b1d4972bfbc136cb2e341e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:47.907120 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_74217fffb3f841cb8d27127268ccdc88" } new_table_name: "default.Table_89a4977bd2c24950a1cb9133ad6a157e" modify_external_catalogs: true
W20260810 07:52:47.907377 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58e123d14b1d4972bfbc136cb2e341e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:47.909384 1064 tablet_bootstrap.cc:654] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:47.927449 868 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1eb27f536648c888a5acc3e940e4c0 (DEFAULT_TABLE table=default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.928665 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1eb27f536648c888a5acc3e940e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.938830 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:47.950443 872 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:47.869697 1047 ts_tablet_manager.cc:1434] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260810 07:52:47.880513 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:47.952639 1047 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:47.951821 1196 raft_consensus.cc:493] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.924793 1154 raft_consensus.cc:493] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:47.954932 1196 raft_consensus.cc:515] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:47.955111 1154 raft_consensus.cc:515] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:47.912935 646 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1eb27f536648c888a5acc3e940e4c0 (DEFAULT_TABLE table=default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:47.956840 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_faede62cb1c04a3ebc3883318c37608d" } modify_external_catalogs: true
I20260810 07:52:47.957438 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_faede62cb1c04a3ebc3883318c37608d" } modify_external_catalogs: true
I20260810 07:52:47.957909 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1eb27f536648c888a5acc3e940e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:47.958513 1154 leader_election.cc:290] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:47.951082 638 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:47.879400 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f679d04625b4c0690a4ce8ea2a5e87b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:47.963272 308 leader_election.cc:304] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:47.964672 1148 raft_consensus.cc:3060] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:47.965350 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet d397310bcfd2470e95fc4b0654749aa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d397310bcfd2470e95fc4b0654749aa4 already in progress: creating tablet
I20260810 07:52:47.965626 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13d8a5248d54b00aa4a47874561745e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:47.965962 1065 tablet_replica.cc:333] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
W20260810 07:52:47.954048 792 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.966080 380 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:47.966503 792 leader_election.cc:304] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:47.966907 1065 raft_consensus.cc:2243] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.967432 1065 raft_consensus.cc:2272] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:47.968374 1075 raft_consensus.cc:2749] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:47.970712 574 leader_election.cc:343] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: e13d8a5248d54b00aa4a47874561745e
I20260810 07:52:47.971959 1065 ts_tablet_manager.cc:1916] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:47.979503 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_74217fffb3f841cb8d27127268ccdc88 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 1 got 0
I20260810 07:52:47.983490 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:47.983547 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:47.984135 896 raft_consensus.cc:2468] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:52:47.985169 573 leader_election.cc:343] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:47.985862 573 leader_election.cc:343] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:47.986021 1071 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:47.986260 573 leader_election.cc:304] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:47.988605 574 leader_election.cc:304] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:47.989563 1154 raft_consensus.cc:2749] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:47.991760 1154 raft_consensus.cc:2804] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:47.992271 1154 raft_consensus.cc:493] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:47.992697 1154 raft_consensus.cc:3060] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:47.993388 1148 raft_consensus.cc:2749] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:47.994115 1196 leader_election.cc:290] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:47.996428 1065 ts_tablet_manager.cc:1929] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:47.997057 1103 tablet_replica.cc:333] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:47.997987 1103 raft_consensus.cc:2243] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:47.998523 1103 raft_consensus.cc:2272] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:48.007275 1154 raft_consensus.cc:515] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.009007 1065 log.cc:1199] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:48.010131 1071 tablet_bootstrap.cc:654] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.010847 1047 tablet_bootstrap.cc:654] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.010224 1065 ts_tablet_manager.cc:1950] T d397310bcfd2470e95fc4b0654749aa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:48.015252 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet d397310bcfd2470e95fc4b0654749aa4 (table default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]) successfully deleted
I20260810 07:52:48.015650 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:48.017360 573 leader_election.cc:343] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.018421 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4772c3359a426e9f105760afff40cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:48.018960 896 raft_consensus.cc:3060] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.027133 896 raft_consensus.cc:2468] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:48.029343 574 leader_election.cc:304] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:48.030249 1196 raft_consensus.cc:2804] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:48.030699 1196 raft_consensus.cc:697] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:48.031489 1196 consensus_queue.cc:237] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.036978 1154 leader_election.cc:290] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:48.046137 1047 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.048240 1047 ts_tablet_manager.cc:1403] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.096s user 0.006s sys 0.005s
I20260810 07:52:48.063264 1047 raft_consensus.cc:359] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.064149 1047 raft_consensus.cc:385] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.064677 1047 raft_consensus.cc:740] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.068356 1047 consensus_queue.cc:260] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.072896 1103 ts_tablet_manager.cc:1916] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:48.079201 1047 ts_tablet_manager.cc:1434] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260810 07:52:48.092413 1047 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.098829 1047 tablet_bootstrap.cc:654] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.117666 1047 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.118248 1047 ts_tablet_manager.cc:1403] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.001s
I20260810 07:52:48.120710 1047 raft_consensus.cc:359] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.127646 1047 raft_consensus.cc:385] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.128059 1047 raft_consensus.cc:740] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.128937 1047 consensus_queue.cc:260] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.131719 1047 ts_tablet_manager.cc:1434] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260810 07:52:48.133052 1047 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.136770 1210 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:48.140296 1214 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:48.150944 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:48.152434 642 tablet_service.cc:1558] Processing DeleteTablet for tablet 0075243274794151ad4e2e0130ae04ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:48.153393 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 99d1bfe95e5d4d8697584b05b28339f3 because the tablet was not found. No further retry: Not found: Tablet not found: 99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:48.157192 1064 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.157653 1064 ts_tablet_manager.cc:1403] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.256s user 0.023s sys 0.033s
I20260810 07:52:48.157945 896 raft_consensus.cc:1275] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:48.162704 1154 consensus_queue.cc:1048] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:52:48.163218 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0075243274794151ad4e2e0130ae04ef because the tablet was not found. No further retry: Not found: Tablet not found: 0075243274794151ad4e2e0130ae04ef
I20260810 07:52:48.165514 417 raft_consensus.cc:1275] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:48.165517 1191 consensus_queue.cc:1048] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:48.168191 1047 tablet_bootstrap.cc:654] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.169034 377 tablet_service.cc:1558] Processing DeleteTablet for tablet f176f0909d064a93ab2384ac037123df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:48.170001 1154 consensus_queue.cc:1048] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:48.182649 1216 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:48.187665 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_ae672329134f4bbf8959c4cc2a85982c" } modify_external_catalogs: true
I20260810 07:52:48.188359 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_ae672329134f4bbf8959c4cc2a85982c" } modify_external_catalogs: true
I20260810 07:52:48.197695 415 raft_consensus.cc:3060] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.206310 1191 consensus_queue.cc:1048] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:48.208724 1071 tablet_bootstrap.cc:492] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:48.209271 1071 ts_tablet_manager.cc:1403] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.358s user 0.010s sys 0.001s
I20260810 07:52:48.215420 1071 raft_consensus.cc:359] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.216329 1071 raft_consensus.cc:385] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.216877 1071 raft_consensus.cc:740] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.218283 1064 raft_consensus.cc:359] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.219166 1064 raft_consensus.cc:385] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.219514 1064 raft_consensus.cc:740] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.220491 1064 consensus_queue.cc:260] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.190775 1154 raft_consensus.cc:493] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.235288 1154 raft_consensus.cc:515] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:48.240052 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f353e1e07485400a964f8e05b0691cc6 consensus state has no leader
I20260810 07:52:48.240767 1154 leader_election.cc:290] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.242908 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:48.244320 573 leader_election.cc:343] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.289832 1228 tablet.cc:1724] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:48.314529 1071 consensus_queue.cc:260] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.317112 1071 ts_tablet_manager.cc:1434] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.108s user 0.007s sys 0.000s
I20260810 07:52:48.318146 1071 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:48.242413 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.318742 897 raft_consensus.cc:2468] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.325462 574 leader_election.cc:304] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:48.338817 1154 raft_consensus.cc:2804] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.339296 1154 raft_consensus.cc:493] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.339704 1154 raft_consensus.cc:3060] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.335934 1210 tablet.cc:1724] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:48.366639 1064 ts_tablet_manager.cc:1434] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.209s user 0.007s sys 0.000s
I20260810 07:52:48.367807 1064 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:48.375140 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:48.377542 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c307f9e0bdd74576bfd1c157610ad9e9" } new_table_name: "default.Table_9f0c5abc8ee0420295dbbbf0f861befa" modify_external_catalogs: true
W20260810 07:52:48.382025 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:48.403024 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69c05d6e49204eedbb305f66dfa2316c consensus state has no leader
I20260810 07:52:48.415660 1154 raft_consensus.cc:515] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.439551 1214 tablet.cc:1724] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:48.444810 1201 mvcc.cc:204] Tried to move back new op lower bound from 7316882915949285376 to 7316882912302915584. Current Snapshot: MvccSnapshot[applied={T|T < 7316882915949285376}]
I20260810 07:52:48.455276 1103 ts_tablet_manager.cc:1929] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:48.461895 1103 log.cc:1199] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:48.463059 1075 raft_consensus.cc:493] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.464684 1075 raft_consensus.cc:515] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.466763 1075 leader_election.cc:290] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:48.462612 1103 ts_tablet_manager.cc:1950] T 99d1bfe95e5d4d8697584b05b28339f3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:48.468695 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.469488 1103 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 25) took 1315 ms. Trace:
I20260810 07:52:48.469887 1103 rpcz_store.cc:276] 0810 07:52:47.154449 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:47.159460 (+ 5011us) service_pool.cc:224] Handling call
0810 07:52:47.996853 (+837393us) ts_tablet_manager.cc:1151] Deleting tablet 99d1bfe95e5d4d8697584b05b28339f3
0810 07:52:47.996925 (+ 72us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:48.206684 (+209759us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:48.455126 (+248442us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:48.469398 (+ 14272us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":646240}
W20260810 07:52:48.471169 792 leader_election.cc:343] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.471513 1103 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 28) took 1076 ms. Trace:
I20260810 07:52:48.471547 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 99d1bfe95e5d4d8697584b05b28339f3 (table default.Table_612e3c4860664a5a964122487ce93bed [id=0d62e87e4971408faa84ce2850fe01e3]) successfully deleted
I20260810 07:52:48.471832 1103 rpcz_store.cc:276] 0810 07:52:47.395444 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:47.395674 (+ 230us) service_pool.cc:224] Handling call
0810 07:52:48.471110 (+1075436us) ts_tablet_manager.cc:1151] Deleting tablet e44b480f5f3d4f239eae2586c1d2001a
0810 07:52:48.471172 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:48.471455 (+ 283us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1074412}
W20260810 07:52:48.473483 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet e44b480f5f3d4f239eae2586c1d2001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e44b480f5f3d4f239eae2586c1d2001a already in progress: creating tablet
W20260810 07:52:48.474737 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet d397310bcfd2470e95fc4b0654749aa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d397310bcfd2470e95fc4b0654749aa4 already in progress: creating tablet
W20260810 07:52:48.477144 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet f176f0909d064a93ab2384ac037123df because the tablet was not found. No further retry: Not found: Tablet not found: f176f0909d064a93ab2384ac037123df
I20260810 07:52:48.490456 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_775fb3e64a80463b920fa697ea14223d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:48.492223 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_775fb3e64a80463b920fa697ea14223d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:48.433501 1196 raft_consensus.cc:493] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.461500 1047 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.469996 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:48.516248 1047 ts_tablet_manager.cc:1403] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.383s user 0.056s sys 0.112s
W20260810 07:52:48.517186 792 leader_election.cc:343] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.517635 792 leader_election.cc:304] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:48.518690 1075 raft_consensus.cc:2749] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:48.519191 1047 raft_consensus.cc:359] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.519814 1047 raft_consensus.cc:385] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.520056 1047 raft_consensus.cc:740] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.516110 1196 raft_consensus.cc:515] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.520684 1047 consensus_queue.cc:260] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.523753 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.523875 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.524477 896 raft_consensus.cc:2468] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.522791 1196 leader_election.cc:290] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:48.525547 574 leader_election.cc:304] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:52:48.528013 573 leader_election.cc:343] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.528412 1196 raft_consensus.cc:2804] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.528764 1196 raft_consensus.cc:493] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.529062 1196 raft_consensus.cc:3060] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.527256 1047 ts_tablet_manager.cc:1434] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260810 07:52:48.534333 1047 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.546080 1196 raft_consensus.cc:515] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.550328 1154 leader_election.cc:290] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.550658 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:48.551301 896 raft_consensus.cc:3060] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.551537 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9369baf30ab64646b7bc042c6da1987b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:48.553083 573 leader_election.cc:343] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.554391 1064 tablet_bootstrap.cc:654] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.557660 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_fce89a2685ea4f499401deb1152927b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:48.558934 896 raft_consensus.cc:2468] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:52:48.559551 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fce89a2685ea4f499401deb1152927b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:48.563745 574 leader_election.cc:304] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:48.579038 1071 tablet_bootstrap.cc:654] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.582182 1154 raft_consensus.cc:2804] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:48.582701 1154 raft_consensus.cc:697] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:48.583490 1154 consensus_queue.cc:237] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.601465 1196 leader_election.cc:290] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:48.602352 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:48.602715 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7626d30f4574658bb226f2e915ee0de" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:48.541847 1047 tablet_bootstrap.cc:654] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.603412 896 raft_consensus.cc:3060] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:48.605347 573 leader_election.cc:343] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.611158 896 raft_consensus.cc:2468] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:48.612437 574 leader_election.cc:304] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:48.614507 1154 raft_consensus.cc:2804] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:48.614986 1154 raft_consensus.cc:697] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:48.615768 1154 consensus_queue.cc:237] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.619261 1064 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.619719 1064 ts_tablet_manager.cc:1403] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.252s user 0.009s sys 0.000s
I20260810 07:52:48.627822 1047 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.628285 1047 ts_tablet_manager.cc:1403] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.094s user 0.010s sys 0.000s
I20260810 07:52:48.630699 1047 raft_consensus.cc:359] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.631354 1047 raft_consensus.cc:385] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.631604 1047 raft_consensus.cc:740] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.622278 1064 raft_consensus.cc:359] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.633610 1064 raft_consensus.cc:385] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.633885 1064 raft_consensus.cc:740] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.634641 1064 consensus_queue.cc:260] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.635109 1047 consensus_queue.cc:260] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.638316 1064 ts_tablet_manager.cc:1434] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260810 07:52:48.639190 1064 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:48.653579 1047 ts_tablet_manager.cc:1434] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260810 07:52:48.654563 1047 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.656569 1064 tablet_bootstrap.cc:654] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.660230 1047 tablet_bootstrap.cc:654] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.670478 1064 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.670997 1064 ts_tablet_manager.cc:1403] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.001s
I20260810 07:52:48.673588 1064 raft_consensus.cc:359] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.674247 1064 raft_consensus.cc:385] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.674494 1064 raft_consensus.cc:740] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.684630 1064 consensus_queue.cc:260] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.687234 1047 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.687681 1047 ts_tablet_manager.cc:1403] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.003s
I20260810 07:52:48.690073 1047 raft_consensus.cc:359] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.690733 1047 raft_consensus.cc:385] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.690995 1047 raft_consensus.cc:740] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.691766 1154 raft_consensus.cc:493] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.692139 1064 ts_tablet_manager.cc:1434] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260810 07:52:48.692166 1154 raft_consensus.cc:515] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.692767 1047 consensus_queue.cc:260] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.693477 1064 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:48.694139 1154 leader_election.cc:290] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.695175 1047 ts_tablet_manager.cc:1434] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260810 07:52:48.696014 1047 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.699571 1064 tablet_bootstrap.cc:654] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.699781 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.700474 896 raft_consensus.cc:2468] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.701541 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:48.704221 573 leader_election.cc:343] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.705494 574 leader_election.cc:304] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:48.707547 1154 raft_consensus.cc:2804] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.707880 1154 raft_consensus.cc:493] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.708173 1154 raft_consensus.cc:3060] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.713341 1071 tablet_bootstrap.cc:492] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:48.713865 1071 ts_tablet_manager.cc:1403] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.396s user 0.017s sys 0.010s
I20260810 07:52:48.715188 1064 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.715660 1064 ts_tablet_manager.cc:1403] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.004s
I20260810 07:52:48.717720 1047 tablet_bootstrap.cc:654] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.718544 1064 raft_consensus.cc:359] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.719298 1064 raft_consensus.cc:385] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.719607 1064 raft_consensus.cc:740] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.720597 1064 consensus_queue.cc:260] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.716166 1154 raft_consensus.cc:515] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.726850 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:48.727435 896 raft_consensus.cc:3060] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.728564 1064 ts_tablet_manager.cc:1434] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260810 07:52:48.730186 1071 raft_consensus.cc:359] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.732206 1064 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:48.735385 1047 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.735812 1047 ts_tablet_manager.cc:1403] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.003s
I20260810 07:52:48.737843 1064 tablet_bootstrap.cc:654] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.738474 896 raft_consensus.cc:2468] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:48.738464 1047 raft_consensus.cc:359] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.739164 1047 raft_consensus.cc:385] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.739459 1047 raft_consensus.cc:740] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.739933 574 leader_election.cc:304] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.740761 1196 raft_consensus.cc:2804] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:48.741202 1196 raft_consensus.cc:697] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:48.741952 1196 consensus_queue.cc:237] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.749215 1064 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.749673 1064 ts_tablet_manager.cc:1403] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20260810 07:52:48.752130 1064 raft_consensus.cc:359] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.752930 1064 raft_consensus.cc:385] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.753206 1064 raft_consensus.cc:740] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.754146 1064 consensus_queue.cc:260] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.755285 1154 leader_election.cc:290] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.756012 1047 consensus_queue.cc:260] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.758364 1047 ts_tablet_manager.cc:1434] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260810 07:52:48.759331 1064 ts_tablet_manager.cc:1434] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260810 07:52:48.759388 1047 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.768252 1064 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:48.774346 1064 tablet_bootstrap.cc:654] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.775117 1047 tablet_bootstrap.cc:654] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.785403 1064 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:48.786237 1064 ts_tablet_manager.cc:1403] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20260810 07:52:48.788931 1064 raft_consensus.cc:359] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.789985 1064 raft_consensus.cc:385] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.790241 1064 raft_consensus.cc:740] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.790858 1064 consensus_queue.cc:260] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.799711 1064 ts_tablet_manager.cc:1434] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260810 07:52:48.799974 1047 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.800099 1196 raft_consensus.cc:493] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.800388 1047 ts_tablet_manager.cc:1403] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.007s
I20260810 07:52:48.800570 1196 raft_consensus.cc:515] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.802768 1196 leader_election.cc:290] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.803210 1075 raft_consensus.cc:493] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.803654 1075 raft_consensus.cc:515] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.804127 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.804756 896 raft_consensus.cc:2468] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.805631 1075 leader_election.cc:290] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:48.806620 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_51a621150c8343f4a0cc9e0e8bb61580" } new_table_name: "default.Table_395605094533475e8e04175ec53c96bb" modify_external_catalogs: true
I20260810 07:52:48.808247 574 leader_election.cc:304] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.809065 1154 raft_consensus.cc:493] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.809470 1154 raft_consensus.cc:515] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.809649 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_4c14a225e70b402d924a5e3de18dacea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:48.810945 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:48.811434 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c14a225e70b402d924a5e3de18dacea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:48.811537 676 raft_consensus.cc:2468] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:48.812870 792 leader_election.cc:304] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.802785 1047 raft_consensus.cc:359] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.813352 1154 leader_election.cc:290] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.813441 1047 raft_consensus.cc:385] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.813724 1075 raft_consensus.cc:2804] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.813755 1047 raft_consensus.cc:740] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.813880 1154 raft_consensus.cc:2804] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.814073 1075 raft_consensus.cc:493] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.814191 1154 raft_consensus.cc:493] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.814388 1075 raft_consensus.cc:3060] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.814494 1154 raft_consensus.cc:3060] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.814405 1047 consensus_queue.cc:260] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.816565 1047 ts_tablet_manager.cc:1434] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260810 07:52:48.817490 1047 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.812554 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.818317 896 raft_consensus.cc:2468] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.819612 574 leader_election.cc:304] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.820521 1198 raft_consensus.cc:2804] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.820887 1198 raft_consensus.cc:493] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.821244 1198 raft_consensus.cc:3060] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.831699 1075 raft_consensus.cc:515] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.833719 1196 raft_consensus.cc:493] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.834195 1196 raft_consensus.cc:515] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:52:48.834770 573 consensus_peers.cc:597] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:48.836265 1196 leader_election.cc:290] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.839006 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:48.839565 676 raft_consensus.cc:3060] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:48.840093 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:48.840658 896 raft_consensus.cc:2468] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:48.832005 1047 tablet_bootstrap.cc:654] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:48.842115 574 leader_election.cc:304] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.846266 676 raft_consensus.cc:2468] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:48.849460 792 leader_election.cc:304] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:48.850135 1196 raft_consensus.cc:2804] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:48.850538 1196 raft_consensus.cc:493] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:48.850955 1196 raft_consensus.cc:3060] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:48.859776 32461 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5195424dd98f444caa75e2ca98d09e6d has no consensus state
I20260810 07:52:48.754320 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08520a2ac0f64c3abd534a32100839a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:48.863068 1071 raft_consensus.cc:385] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.807334 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.863463 1071 raft_consensus.cc:740] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.808184 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.817487 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.837148 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:48.838490 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.865437 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_1992a5b4d5144b85986cec15a6b24446 [id=2e56b185007f4d798dba6e17d0b14818]). Expected version 1 got 0
I20260810 07:52:48.866194 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_89a4977bd2c24950a1cb9133ad6a157e [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 2 got 0
I20260810 07:52:48.867224 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:48.867910 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
W20260810 07:52:48.869130 573 leader_election.cc:343] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:48.870103 573 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:48.870954 573 leader_election.cc:343] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:48.871771 573 leader_election.cc:343] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.861967 1198 raft_consensus.cc:515] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.873684 376 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 (DEFAULT_TABLE table=default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:48.874866 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba026e367ea487fb0dc8aef5d5db606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:48.876145 642 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 (DEFAULT_TABLE table=default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:48.877396 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba026e367ea487fb0dc8aef5d5db606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:48.881167 871 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 (DEFAULT_TABLE table=default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:48.881910 377 tablet_service.cc:1511] Processing CreateTablet for tablet 5195424dd98f444caa75e2ca98d09e6d (DEFAULT_TABLE table=default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:48.882457 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba026e367ea487fb0dc8aef5d5db606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:48.883059 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5195424dd98f444caa75e2ca98d09e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:48.894961 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:48.895596 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:48.896286 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_c307f9e0bdd74576bfd1c157610ad9e9 [id=8ebf95685c034249a05157e5d113892d]). Expected version 1 got 0
I20260810 07:52:48.896837 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]). Expected version 2 got 0
I20260810 07:52:48.897626 32462 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_1992a5b4d5144b85986cec15a6b24446 [id=2e56b185007f4d798dba6e17d0b14818] alter complete (version 1)
W20260810 07:52:48.898383 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08520a2ac0f64c3abd534a32100839a4 consensus state has no leader
I20260810 07:52:48.899530 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:48.900920 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 99d1bfe95e5d4d8697584b05b28339f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:48.903126 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:48.903126 638 tablet_service.cc:1558] Processing DeleteTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:48.904770 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 99d1bfe95e5d4d8697584b05b28339f3 because the tablet was not found. No further retry: Not found: Tablet not found: 99d1bfe95e5d4d8697584b05b28339f3
I20260810 07:52:48.874159 1154 raft_consensus.cc:515] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.906858 646 tablet_service.cc:1558] Processing DeleteTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:48 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:48.909554 1263 tablet_replica.cc:333] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:48.910462 1263 raft_consensus.cc:2243] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:48.910975 1263 raft_consensus.cc:2272] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:48.912040 869 tablet_service.cc:1558] Processing DeleteTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:48 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:48.912673 382 tablet_service.cc:1558] Processing DeleteTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:48.913427 872 tablet_service.cc:1558] Processing DeleteTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:48.913663 382 tablet_service.cc:1558] Processing DeleteTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:48 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:48.913074 1071 consensus_queue.cc:260] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.914305 1103 tablet_replica.cc:333] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:48.915599 1103 raft_consensus.cc:2243] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:48.916152 1103 raft_consensus.cc:2272] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:48.917793 1263 ts_tablet_manager.cc:1916] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:48.919589 32460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 13) took 1493 ms (client timeout 1000 ms). Trace:
I20260810 07:52:48.922922 1103 ts_tablet_manager.cc:1916] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:48.920032 32460 rpcz_store.cc:269] 0810 07:52:47.426523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:47.427600 (+ 1077us) service_pool.cc:224] Handling call
0810 07:52:47.484572 (+ 56972us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:48.919479 (+1434907us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:48.003002 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:48.003399 (+ 397us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:48.003423 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:52:48.003434 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:48.003714 (+ 280us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:48.003753 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:48.003776 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:48.017905 (+ 14129us) tablet.cc:701] Row locks acquired
0810 07:52:48.017942 (+ 37us) write_op.cc:260] PREPARE: finished
0810 07:52:48.018088 (+ 146us) write_op.cc:270] Start()
0810 07:52:48.018183 (+ 95us) write_op.cc:276] Timestamp: P: 1786348368017924 usec, L: 0
0810 07:52:48.018198 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:48.018760 (+ 562us) log.cc:844] Serialized 532 byte log entry
0810 07:52:48.091800 (+ 73040us) op_driver.cc:464] REPLICATION: finished
0810 07:52:48.094015 (+ 2215us) write_op.cc:301] APPLY: starting
0810 07:52:48.094084 (+ 69us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:48.094181 (+ 97us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:48.094293 (+ 112us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:48.094471 (+ 178us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:48.094628 (+ 157us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:48.094688 (+ 60us) write_op.cc:312] APPLY: finished
0810 07:52:48.095050 (+ 362us) log.cc:844] Serialized 22 byte log entry
0810 07:52:48.095233 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:48.095300 (+ 67us) write_op.cc:454] Released schema lock
0810 07:52:48.095335 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:48.095615 (+ 280us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:48.095639 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":125525760,"child_traces":[["op",{"apply.queue_time_us":2022,"apply.run_cpu_time_us":1567,"apply.run_wall_time_us":1567,"num_ops":1,"prepare.queue_time_us":206909,"prepare.run_cpu_time_us":2366,"prepare.run_wall_time_us":36735,"raft.queue_time_us":1659,"raft.run_cpu_time_us":1130,"raft.run_wall_time_us":1129,"replication_time_us":73482}]]}
I20260810 07:52:48.928956 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:48.931435 573 leader_election.cc:343] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.937001 1198 leader_election.cc:290] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.937698 1260 raft_consensus.cc:493] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:48.938359 1260 raft_consensus.cc:515] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.940547 1071 ts_tablet_manager.cc:1434] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.226s user 0.006s sys 0.000s
I20260810 07:52:48.947400 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:48.948465 638 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:48.949554 573 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.947796 1071 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:48.951913 1154 leader_election.cc:290] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:48.959404 1260 leader_election.cc:290] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:48.961529 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:48.950002 1259 raft_consensus.cc:2804] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:48.962967 1047 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:48.963407 1259 raft_consensus.cc:697] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:48.963976 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:48.964383 1259 consensus_queue.cc:237] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:48.966878 1047 ts_tablet_manager.cc:1403] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.150s user 0.019s sys 0.017s
I20260810 07:52:48.951535 1265 tablet_replica.cc:333] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:48.966969 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38193
--webserver_interface=127.31.139.126
--webserver_port=38367
--builtin_ntp_servers=127.31.139.84:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:52:48.968787 1265 raft_consensus.cc:2243] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
W20260810 07:52:48.950445 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 101078us
I20260810 07:52:48.969614 1047 raft_consensus.cc:359] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.970316 1047 raft_consensus.cc:385] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:48.970606 1047 raft_consensus.cc:740] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:48.970515 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fcae5b2e5448bf89b1c0587f46ad40" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:48.975289 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f09a245fe4c1ea4e391166eaf3d3d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:52:48.976176 792 leader_election.cc:343] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:48.976931 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 16) took 783 ms (client timeout 1000 ms). Trace:
I20260810 07:52:48.977062 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:52:48.977150 792 leader_election.cc:343] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:48.977589 896 raft_consensus.cc:3060] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:48.977763 792 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:48.978854 32463 rpcz_store.cc:269] 0810 07:52:48.193205 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:48.193353 (+ 148us) service_pool.cc:224] Handling call
0810 07:52:48.976859 (+783506us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7}
I20260810 07:52:48.984570 1047 consensus_queue.cc:260] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:48.986634 1047 ts_tablet_manager.cc:1434] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260810 07:52:48.987007 1103 ts_tablet_manager.cc:1929] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:48.987377 1103 log.cc:1199] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:48.987509 1047 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:48.988209 1103 ts_tablet_manager.cc:1950] T d6f37bba532b4d8bbc00e6d91f4c1820 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:48.989598 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]) successfully deleted
I20260810 07:52:48.990486 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fcae5b2e5448bf89b1c0587f46ad40" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:48.991729 1075 leader_election.cc:290] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:48.992311 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet a56f09a245fe4c1ea4e391166eaf3d3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a56f09a245fe4c1ea4e391166eaf3d3d already in progress: creating tablet
I20260810 07:52:48.993610 1265 raft_consensus.cc:2272] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:48.995617 1265 ts_tablet_manager.cc:1916] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.001360 638 tablet_service.cc:1511] Processing CreateTablet for tablet 5195424dd98f444caa75e2ca98d09e6d (DEFAULT_TABLE table=default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.002266 868 tablet_service.cc:1511] Processing CreateTablet for tablet 5195424dd98f444caa75e2ca98d09e6d (DEFAULT_TABLE table=default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.002655 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5195424dd98f444caa75e2ca98d09e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.003363 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5195424dd98f444caa75e2ca98d09e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.002866 896 raft_consensus.cc:2468] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:49.006356 869 tablet_service.cc:1511] Processing CreateTablet for tablet acf740de7abb40bfabfc3a7859cc159b (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.007372 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf740de7abb40bfabfc3a7859cc159b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.009316 382 tablet_service.cc:1511] Processing CreateTablet for tablet acf740de7abb40bfabfc3a7859cc159b (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.010324 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf740de7abb40bfabfc3a7859cc159b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.010151 646 tablet_service.cc:1511] Processing CreateTablet for tablet acf740de7abb40bfabfc3a7859cc159b (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.011317 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf740de7abb40bfabfc3a7859cc159b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:49.013273 792 consensus_peers.cc:597] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:52:49.013935 792 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.014302 792 leader_election.cc:304] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.015038 1075 raft_consensus.cc:2749] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:49.016381 574 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.016870 574 leader_election.cc:304] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:49.017477 1263 ts_tablet_manager.cc:1929] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:49.017928 1263 log.cc:1199] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:49.018836 1263 ts_tablet_manager.cc:1950] T d6f37bba532b4d8bbc00e6d91f4c1820 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:49.020409 1263 tablet_replica.cc:333] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:49.021523 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]) successfully deleted
I20260810 07:52:49.023605 1198 raft_consensus.cc:2749] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.033195 1263 raft_consensus.cc:2243] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.033787 1263 raft_consensus.cc:2272] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.035485 1071 tablet_bootstrap.cc:654] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.035749 1263 ts_tablet_manager.cc:1916] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:49.046353 792 consensus_peers.cc:597] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6f37bba532b4d8bbc00e6d91f4c1820. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:49.041443 574 leader_election.cc:304] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:49.051015 1273 raft_consensus.cc:2804] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:49.060158 32470 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_89a4977bd2c24950a1cb9133ad6a157e [id=57fbc9f2436f42289d66a57eec7232a9] alter complete (version 2)
I20260810 07:52:49.066015 872 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:49.068172 1064 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:49.072659 872 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.073659 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.080241 1064 tablet_bootstrap.cc:654] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.092549 1265 ts_tablet_manager.cc:1929] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:49.094017 868 tablet_service.cc:1511] Processing CreateTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 (DEFAULT_TABLE table=default.Table_3a887c70a4ef45ccace031aa6fa907de [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.094094 377 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.095149 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3675c5bae7a42769223ef5b2fc433b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.095177 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.097543 376 tablet_service.cc:1558] Processing DeleteTablet for tablet e13d8a5248d54b00aa4a47874561745e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:46 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:49.098312 1103 tablet_replica.cc:333] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:49.099058 1103 raft_consensus.cc:2243] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.099560 1103 raft_consensus.cc:2272] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.093032 1265 log.cc:1199] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:49.099923 1071 tablet_bootstrap.cc:492] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:49.100359 1071 ts_tablet_manager.cc:1403] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.157s user 0.014s sys 0.000s
I20260810 07:52:49.100978 1265 ts_tablet_manager.cc:1950] T a56f09a245fe4c1ea4e391166eaf3d3d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:49.101696 1103 ts_tablet_manager.cc:1916] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.102950 1071 raft_consensus.cc:359] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.103587 1071 raft_consensus.cc:385] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.103976 1071 raft_consensus.cc:740] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.104633 1071 consensus_queue.cc:260] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.107331 1064 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:49.107453 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet a56f09a245fe4c1ea4e391166eaf3d3d (table default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]) successfully deleted
I20260810 07:52:49.107769 1064 ts_tablet_manager.cc:1403] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.000s
I20260810 07:52:49.109335 1265 tablet_replica.cc:333] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:49.110370 1064 raft_consensus.cc:359] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.111044 1064 raft_consensus.cc:385] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.111321 1064 raft_consensus.cc:740] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.112740 376 tablet_service.cc:1558] Processing DeleteTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:49.112689 1064 consensus_queue.cc:260] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.112711 1265 raft_consensus.cc:2243] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:49.114682 376 tablet_service.cc:1511] Processing CreateTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 (DEFAULT_TABLE table=default.Table_3a887c70a4ef45ccace031aa6fa907de [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.115805 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3675c5bae7a42769223ef5b2fc433b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.117125 1064 ts_tablet_manager.cc:1434] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260810 07:52:49.118198 1265 pending_rounds.cc:70] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Trying to abort 1 pending ops.
I20260810 07:52:49.121727 1265 pending_rounds.cc:77] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316882919244648448 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "d6f37bba532b4d8bbc00e6d91f4c1820" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_faede62cb1c04a3ebc3883318c37608d" dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" new_extra_config { } }
I20260810 07:52:49.124565 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:49.125283 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
W20260810 07:52:49.125028 1265 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55008: Aborted: Op aborted
I20260810 07:52:49.127414 1071 ts_tablet_manager.cc:1434] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260810 07:52:49.130442 1064 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:49.131317 32470 catalog_manager.cc:5719] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:49.129839 1265 raft_consensus.cc:2272] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.132694 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]). Expected version 2 got 0
I20260810 07:52:49.133497 1071 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:49.134068 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_5eac9bcc1944426bac3bdacfcb745f39 [id=1083d105acd24b249933790d21f43ef5]). Expected version 1 got 0
W20260810 07:52:49.134904 1256 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55008: Illegal state: RaftConsensus is not running: State = Stopped
I20260810 07:52:49.134928 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:49.137424 1047 tablet_bootstrap.cc:654] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.140851 1071 tablet_bootstrap.cc:654] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.155378 1154 consensus_queue.cc:1048] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.145700 1273 raft_consensus.cc:697] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:49.162037 1265 ts_tablet_manager.cc:1916] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.161957 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:49.163398 1064 tablet_bootstrap.cc:654] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.182293 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_97132925a8444a3d9c702a3ae7cbb209"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:49.184240 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97132925a8444a3d9c702a3ae7cbb209 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.184619 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_1992a5b4d5144b85986cec15a6b24446" } new_table_name: "default.Table_643b0093b7db404e866459a579b26ac0" modify_external_catalogs: true
I20260810 07:52:49.185137 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_452dd2adccb043c685e5eb8af81bd4db" } modify_external_catalogs: true
I20260810 07:52:49.186167 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d73cd38f1d774b448a86e7e8f5b10242"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:49.187945 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d73cd38f1d774b448a86e7e8f5b10242 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.189129 32461 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0741e2ea30224c6180516e0791c77331" } modify_external_catalogs: true
I20260810 07:52:49.190299 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_680195385fd2411baf85a6a49bab6feb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:49.192109 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_680195385fd2411baf85a6a49bab6feb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.194046 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_452dd2adccb043c685e5eb8af81bd4db" } modify_external_catalogs: true
I20260810 07:52:49.195663 32461 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0741e2ea30224c6180516e0791c77331" } modify_external_catalogs: true
I20260810 07:52:49.161315 1273 consensus_queue.cc:237] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.205003 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_29c70477feb94d599ff0c03f584bc04b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:49.205519 638 tablet_service.cc:1511] Processing CreateTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:49.206588 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:49.206812 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29c70477feb94d599ff0c03f584bc04b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.044544 1196 raft_consensus.cc:515] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.210237 1196 leader_election.cc:290] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:49.210362 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_aeb2755859d043bfa96d14a40dddb4be" } new_table_name: "default.Table_f89e0d36f8cb41f58f17e017b9172a83" modify_external_catalogs: true
I20260810 07:52:49.211714 1047 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:49.212139 1047 ts_tablet_manager.cc:1403] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.225s user 0.010s sys 0.005s
I20260810 07:52:49.215673 1047 raft_consensus.cc:359] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.219692 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:49.220093 32460 master_service.cc:960] ReplaceTablet: received request to replace tablet 6b5eed268ee545ee91420cd792608b23 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:49.221419 1071 tablet_bootstrap.cc:492] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:49.221814 1071 ts_tablet_manager.cc:1403] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.089s user 0.016s sys 0.002s
I20260810 07:52:49.224233 1071 raft_consensus.cc:359] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.224849 1071 raft_consensus.cc:385] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.225085 1071 raft_consensus.cc:740] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.237867 1071 consensus_queue.cc:260] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.246145 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44b480f5f3d4f239eae2586c1d2001a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:49.246840 896 raft_consensus.cc:3060] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.247869 1265 ts_tablet_manager.cc:1929] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:49.248277 1265 log.cc:1199] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:49.251044 1265 ts_tablet_manager.cc:1950] T d6f37bba532b4d8bbc00e6d91f4c1820 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:49.218037 644 tablet_service.cc:1511] Processing CreateTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 (DEFAULT_TABLE table=default.Table_3a887c70a4ef45ccace031aa6fa907de [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:49.229418 573 leader_election.cc:343] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.256167 1071 ts_tablet_manager.cc:1434] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260810 07:52:49.256150 1047 raft_consensus.cc:385] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.256572 1047 raft_consensus.cc:740] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.263810 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3675c5bae7a42769223ef5b2fc433b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:49.265504 1071 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:49.270670 1064 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:49.270803 1071 tablet_bootstrap.cc:654] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.271093 1064 ts_tablet_manager.cc:1403] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.142s user 0.013s sys 0.008s
I20260810 07:52:49.273581 1064 raft_consensus.cc:359] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.274189 1064 raft_consensus.cc:385] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.274433 1064 raft_consensus.cc:740] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.276978 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet d6f37bba532b4d8bbc00e6d91f4c1820 (table default.Table_faede62cb1c04a3ebc3883318c37608d [id=a1b55d4a058a4d43bbbbce82198aa6ef]) successfully deleted
I20260810 07:52:49.281575 1263 ts_tablet_manager.cc:1929] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:49.288076 1064 consensus_queue.cc:260] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.289201 1265 tablet_replica.cc:333] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:49.290469 1265 raft_consensus.cc:2243] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.290939 1265 pending_rounds.cc:70] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Trying to abort 1 pending ops.
I20260810 07:52:49.291175 1265 pending_rounds.cc:77] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882915468341248 op_type: NO_OP noop_request { }
I20260810 07:52:49.291539 1265 raft_consensus.cc:2889] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:49.291920 1265 raft_consensus.cc:2272] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.303380 1064 ts_tablet_manager.cc:1434] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.032s user 0.001s sys 0.003s
I20260810 07:52:49.305630 1071 tablet_bootstrap.cc:492] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:49.306108 1071 ts_tablet_manager.cc:1403] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.001s
I20260810 07:52:49.308611 1064 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:49.308511 1071 raft_consensus.cc:359] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.312762 1071 raft_consensus.cc:385] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.313105 1071 raft_consensus.cc:740] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.314694 1263 log.cc:1199] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:49.284291 1047 consensus_queue.cc:260] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.315873 1265 ts_tablet_manager.cc:1916] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.316888 1071 consensus_queue.cc:260] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.319041 1047 ts_tablet_manager.cc:1434] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260810 07:52:49.319996 1047 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:49.325433 1071 ts_tablet_manager.cc:1434] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260810 07:52:49.332151 1064 tablet_bootstrap.cc:654] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.333904 1263 ts_tablet_manager.cc:1950] T a56f09a245fe4c1ea4e391166eaf3d3d P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:49.340195 1071 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:49.341174 1103 ts_tablet_manager.cc:1929] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:49.351464 1263 tablet_replica.cc:333] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:49.352048 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet a56f09a245fe4c1ea4e391166eaf3d3d (table default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]) successfully deleted
I20260810 07:52:49.352898 1071 tablet_bootstrap.cc:654] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.361407 1064 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:49.361920 1064 ts_tablet_manager.cc:1403] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.006s
I20260810 07:52:49.365181 1064 raft_consensus.cc:359] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.365990 1064 raft_consensus.cc:385] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.366293 1064 raft_consensus.cc:740] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.366936 1064 consensus_queue.cc:260] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.371132 1265 ts_tablet_manager.cc:1929] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:49.373322 1265 log.cc:1199] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/0b4772c3359a426e9f105760afff40cb
I20260810 07:52:49.374092 896 raft_consensus.cc:2468] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:49.369233 1196 consensus_queue.cc:1048] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.376005 574 leader_election.cc:304] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:49.381940 1263 raft_consensus.cc:2243] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.382627 1263 raft_consensus.cc:2272] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.383234 1064 ts_tablet_manager.cc:1434] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.021s user 0.004s sys 0.002s
I20260810 07:52:49.374331 1265 ts_tablet_manager.cc:1950] T 0b4772c3359a426e9f105760afff40cb P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:49.383915 1196 raft_consensus.cc:2804] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:49.384502 1196 raft_consensus.cc:697] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:49.385454 1259 raft_consensus.cc:493] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.385628 1196 consensus_queue.cc:237] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.386171 1259 raft_consensus.cc:515] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.390410 1259 leader_election.cc:290] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:49.392494 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.393090 1047 tablet_bootstrap.cc:654] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:49.394380 792 leader_election.cc:343] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.396178 1071 tablet_bootstrap.cc:492] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:49.407737 1064 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:49.408215 1103 log.cc:1199] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/e13d8a5248d54b00aa4a47874561745e
I20260810 07:52:49.409338 1103 ts_tablet_manager.cc:1950] T e13d8a5248d54b00aa4a47874561745e P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:49.412747 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet e13d8a5248d54b00aa4a47874561745e (table default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]) successfully deleted
I20260810 07:52:49.419370 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 0b4772c3359a426e9f105760afff40cb (table default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]) successfully deleted
W20260810 07:52:49.423781 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7a1eb27f536648c888a5acc3e940e4c0 consensus state has no leader
I20260810 07:52:49.423980 1071 ts_tablet_manager.cc:1403] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.000s
I20260810 07:52:49.426385 1265 tablet_replica.cc:333] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:49.428223 1265 raft_consensus.cc:2243] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:52:49.447131 574 consensus_peers.cc:597] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b4772c3359a426e9f105760afff40cb. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:49.448364 1047 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:49.448918 1047 ts_tablet_manager.cc:1403] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.129s user 0.007s sys 0.005s
I20260810 07:52:49.434329 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.453523 1263 ts_tablet_manager.cc:1916] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.470599 32467 master_service.cc:960] ReplaceTablet: received request to replace tablet e44b480f5f3d4f239eae2586c1d2001a from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:49.474025 1265 raft_consensus.cc:2272] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.482245 1257 raft_consensus.cc:493] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.483062 1257 raft_consensus.cc:515] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.489878 1071 raft_consensus.cc:359] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.491448 1314 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:49.490628 1071 raft_consensus.cc:385] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.494261 1071 raft_consensus.cc:740] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.494999 1071 consensus_queue.cc:260] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.498399 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.497577 1257 leader_election.cc:290] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:49.500052 792 leader_election.cc:343] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.501673 1265 ts_tablet_manager.cc:1916] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.508002 1064 tablet_bootstrap.cc:654] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.512866 1071 ts_tablet_manager.cc:1434] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.089s user 0.010s sys 0.000s
I20260810 07:52:49.538785 1071 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:49.541337 792 consensus_peers.cc:597] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:52:49.542343 1288 log.cc:927] Time spent T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260810 07:52:49.547842 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_5eac9bcc1944426bac3bdacfcb745f39" } new_table_name: "default.Table_8880a03671844a10931425b1c422e0da" modify_external_catalogs: true
I20260810 07:52:49.559180 1265 ts_tablet_manager.cc:1929] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:49.559690 1265 log.cc:1199] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/e44b480f5f3d4f239eae2586c1d2001a
I20260810 07:52:49.560525 1265 ts_tablet_manager.cc:1950] T e44b480f5f3d4f239eae2586c1d2001a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:49.560873 1260 raft_consensus.cc:493] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.561429 1260 raft_consensus.cc:515] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.565472 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.566483 1071 tablet_bootstrap.cc:654] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:49.567066 792 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.567781 1064 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:49.568184 1064 ts_tablet_manager.cc:1403] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.161s user 0.013s sys 0.005s
I20260810 07:52:49.571597 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet e44b480f5f3d4f239eae2586c1d2001a (table default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]) successfully deleted
I20260810 07:52:49.573148 1064 raft_consensus.cc:359] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.573882 1064 raft_consensus.cc:385] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.415836 1300 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260810 07:52:49.578337 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet d6f37bba532b4d8bbc00e6d91f4c1820 because the tablet was not found. No further retry: Not found: Tablet not found: d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:49.574151 1064 raft_consensus.cc:740] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.580484 1103 tablet_replica.cc:333] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:49.579993 1064 consensus_queue.cc:260] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.581295 1103 raft_consensus.cc:2243] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.581809 1103 raft_consensus.cc:2272] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.585285 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.589746 411 raft_consensus.cc:3060] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.567055 1260 leader_election.cc:290] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:49.593632 1103 ts_tablet_manager.cc:1916] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.594605 1064 ts_tablet_manager.cc:1434] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.026s user 0.000s sys 0.007s
I20260810 07:52:49.603448 1273 raft_consensus.cc:493] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.604054 1273 raft_consensus.cc:515] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.606199 1064 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:49.606633 1047 raft_consensus.cc:359] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.607272 1047 raft_consensus.cc:385] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.607544 1047 raft_consensus.cc:740] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:52:49.612020 792 leader_election.cc:343] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.612565 792 leader_election.cc:304] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.614753 1260 raft_consensus.cc:2749] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:49.615393 792 leader_election.cc:343] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.616080 792 leader_election.cc:304] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.624478 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.632018 1064 tablet_bootstrap.cc:654] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.635739 1260 raft_consensus.cc:2749] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.636487 1260 raft_consensus.cc:493] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.637034 1260 raft_consensus.cc:515] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.639424 1260 leader_election.cc:290] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:49.642313 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.608119 1047 consensus_queue.cc:260] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.608323 1198 consensus_queue.cc:1048] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.648476 1047 ts_tablet_manager.cc:1434] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.199s user 0.006s sys 0.000s
I20260810 07:52:49.648761 1289 raft_consensus.cc:493] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.649549 1047 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:49.649437 1289 raft_consensus.cc:515] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.611335 1154 raft_consensus.cc:493] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.653328 1154 raft_consensus.cc:515] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.654477 1064 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:49.654938 1064 ts_tablet_manager.cc:1403] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.008s
I20260810 07:52:49.655472 1154 leader_election.cc:290] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:52:49.613368 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.657296 677 raft_consensus.cc:2468] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:49.617882 676 raft_consensus.cc:1275] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Refusing update from remote peer 9e545e0597e24d61b7265abed2ccdcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:52:49.658008 1064 raft_consensus.cc:359] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.658720 1064 raft_consensus.cc:385] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.659016 1064 raft_consensus.cc:740] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.659713 1064 consensus_queue.cc:260] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.662045 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:49.625993 1273 leader_election.cc:290] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:49.662772 896 raft_consensus.cc:2468] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:49.663100 1289 leader_election.cc:290] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:49.627007 573 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.662248 1103 ts_tablet_manager.cc:1929] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:49.645514 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.666846 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_15f51029cf1f4d2a8576e3c2e28ae4a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:49.667399 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d397310bcfd2470e95fc4b0654749aa4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.667999 412 raft_consensus.cc:3060] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:49.668848 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15f51029cf1f4d2a8576e3c2e28ae4a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.668805 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:49.670909 573 leader_election.cc:343] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:49.671664 573 leader_election.cc:343] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.666620 1103 log.cc:1199] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/0b4772c3359a426e9f105760afff40cb
I20260810 07:52:49.681555 1103 ts_tablet_manager.cc:1950] T 0b4772c3359a426e9f105760afff40cb P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:49.681149 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_128834a6bbbe4f6da862be15162b8562"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:49.683120 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_128834a6bbbe4f6da862be15162b8562 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.685038 1302 tablet.cc:1724] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:49.689878 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 0b4772c3359a426e9f105760afff40cb (table default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]) successfully deleted
I20260810 07:52:49.693132 1289 consensus_queue.cc:1048] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.698644 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:49.699703 895 raft_consensus.cc:2468] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:49.700928 1148 raft_consensus.cc:493] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.701722 574 leader_election.cc:304] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:49.701581 1148 raft_consensus.cc:515] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.702277 792 leader_election.cc:304] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
W20260810 07:52:49.703436 574 leader_election.cc:343] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:49.704037 1260 raft_consensus.cc:2804] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:49.704381 1260 raft_consensus.cc:493] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:49.703907 574 leader_election.cc:304] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:49.704758 1260 raft_consensus.cc:3060] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.705255 1273 raft_consensus.cc:2804] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:49.705919 1273 raft_consensus.cc:493] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:49.706398 1273 raft_consensus.cc:3060] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.707804 1154 raft_consensus.cc:2749] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.708812 1154 consensus_queue.cc:1048] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.709450 574 leader_election.cc:304] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:49.710975 1154 raft_consensus.cc:2804] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:49.711580 1154 raft_consensus.cc:493] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:49.712073 1154 raft_consensus.cc:3060] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.713883 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:49.717295 895 raft_consensus.cc:2393] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
W20260810 07:52:49.718111 792 leader_election.cc:343] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.717911 1257 consensus_queue.cc:1048] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:49.704607 1148 leader_election.cc:290] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:49.720242 792 leader_election.cc:343] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.720795 792 leader_election.cc:304] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.727222 1266 raft_consensus.cc:2749] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.728325 1064 ts_tablet_manager.cc:1434] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.073s user 0.007s sys 0.001s
I20260810 07:52:49.729218 1273 raft_consensus.cc:515] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.733350 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:49.734001 1273 leader_election.cc:290] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:49.737216 1260 raft_consensus.cc:515] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.740903 1260 leader_election.cc:290] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:49.742430 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:49.738905 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:49.745431 895 raft_consensus.cc:2393] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
W20260810 07:52:49.741967 573 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:49.744537 792 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.753707 574 leader_election.cc:304] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:49.756381 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b76191c5bed41509344376f1810e7aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.757349 1196 raft_consensus.cc:2749] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.758726 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1373ebbd3ba4262806bfc17823d25b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:49.759676 678 raft_consensus.cc:2393] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:49.765461 792 leader_election.cc:304] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.770913 1289 consensus_queue.cc:1048] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:49.771250 1260 raft_consensus.cc:2749] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.779662 308 leader_election.cc:304] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.782495 1331 raft_consensus.cc:3060] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:49.799288 1331 raft_consensus.cc:2749] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:49.819276 1071 tablet_bootstrap.cc:492] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:49.819749 1071 ts_tablet_manager.cc:1403] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.281s user 0.015s sys 0.025s
I20260810 07:52:49.822752 1071 raft_consensus.cc:359] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.824748 1071 raft_consensus.cc:385] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:49.825037 1071 raft_consensus.cc:740] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:49.839286 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 6b5eed268ee545ee91420cd792608b23
I20260810 07:52:49.841291 32460 catalog_manager.cc:6807] ReplaceTablet: tablet 6b5eed268ee545ee91420cd792608b23 deleted and replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658
I20260810 07:52:49.843547 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3d59e31692ec40109b7e69c59019dd83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:49.846487 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5eed268ee545ee91420cd792608b23 with delete_type TABLET_DATA_DELETED (replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:49.847213 1265 tablet_replica.cc:333] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:49.848558 1265 raft_consensus.cc:2243] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:49.849510 1265 raft_consensus.cc:2272] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.852190 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5eed268ee545ee91420cd792608b23 with delete_type TABLET_DATA_DELETED (replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:49.854238 1260 raft_consensus.cc:493] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:49.854771 1260 raft_consensus.cc:515] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.862089 1260 leader_election.cc:290] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:49.844872 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:49.863337 1103 tablet_replica.cc:333] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:49.864131 1103 raft_consensus.cc:2243] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:49.864761 1103 raft_consensus.cc:2272] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:49.865381 1265 ts_tablet_manager.cc:1916] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.867219 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:49.867398 1103 ts_tablet_manager.cc:1916] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:49.869870 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3523dbadfd954f2da7cd8e456fffa4c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:49.871033 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:49.868470 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d59e31692ec40109b7e69c59019dd83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:49.872933 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3523dbadfd954f2da7cd8e456fffa4c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:49.890290 32467 master_service.cc:960] ReplaceTablet: received request to replace tablet a1373ebbd3ba4262806bfc17823d25b5 from {username='slave'} at 127.0.0.1:43806
W20260810 07:52:49.885658 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69c05d6e49204eedbb305f66dfa2316c consensus state has no leader
I20260810 07:52:49.893220 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:49.895115 1103 ts_tablet_manager.cc:1929] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
W20260810 07:52:49.898478 792 leader_election.cc:343] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:49.899801 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 15) took 818 ms (client timeout 1000 ms). Trace:
I20260810 07:52:49.894291 1071 consensus_queue.cc:260] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:49.902153 1103 log.cc:1199] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/6b5eed268ee545ee91420cd792608b23
I20260810 07:52:49.919020 1103 ts_tablet_manager.cc:1950] T 6b5eed268ee545ee91420cd792608b23 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:49.922899 1071 ts_tablet_manager.cc:1434] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.103s user 0.006s sys 0.000s
I20260810 07:52:49.925462 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_643b0093b7db404e866459a579b26ac0 [id=2e56b185007f4d798dba6e17d0b14818]) successfully deleted
W20260810 07:52:49.900233 32470 rpcz_store.cc:269] 0810 07:52:49.081552 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.084037 (+ 2485us) service_pool.cc:224] Handling call
0810 07:52:49.899735 (+815698us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:49.224941 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:49.225335 (+ 394us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:49.225358 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:52:49.225370 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:49.225632 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:49.225673 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:49.225696 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:49.225847 (+ 151us) tablet.cc:701] Row locks acquired
0810 07:52:49.225860 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:49.225974 (+ 114us) write_op.cc:270] Start()
0810 07:52:49.226057 (+ 83us) write_op.cc:276] Timestamp: P: 1786348369225903 usec, L: 0
0810 07:52:49.226073 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:52:49.226565 (+ 492us) log.cc:844] Serialized 532 byte log entry
0810 07:52:49.289191 (+ 62626us) op_driver.cc:464] REPLICATION: finished
0810 07:52:49.290147 (+ 956us) write_op.cc:301] APPLY: starting
0810 07:52:49.290211 (+ 64us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:49.290331 (+ 120us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:49.290429 (+ 98us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:49.290590 (+ 161us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:49.290723 (+ 133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:49.290771 (+ 48us) write_op.cc:312] APPLY: finished
0810 07:52:49.290950 (+ 179us) log.cc:844] Serialized 22 byte log entry
0810 07:52:49.291136 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:49.291204 (+ 68us) write_op.cc:454] Released schema lock
0810 07:52:49.291238 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:49.291480 (+ 242us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:49.291505 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3185,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":767,"apply.run_cpu_time_us":1299,"apply.run_wall_time_us":1299,"num_ops":1,"prepare.queue_time_us":88392,"prepare.run_cpu_time_us":1961,"prepare.run_wall_time_us":1960,"raft.queue_time_us":5023,"raft.run_cpu_time_us":975,"raft.run_wall_time_us":976,"replication_time_us":63017}]]}
I20260810 07:52:49.927304 1071 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:49.931795 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 95fcae5b2e5448bf89b1c0587f46ad40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95fcae5b2e5448bf89b1c0587f46ad40 already in progress: creating tablet
I20260810 07:52:49.940213 1047 tablet_bootstrap.cc:654] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.942157 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5eed268ee545ee91420cd792608b23 with delete_type TABLET_DATA_DELETED (replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:49.943117 644 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:49.955849 1071 tablet_bootstrap.cc:654] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:49.968811 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:49.936435 1154 raft_consensus.cc:515] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:49.974604 1154 leader_election.cc:290] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:49.976375 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:49.983812 792 leader_election.cc:343] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:49.984237 792 leader_election.cc:304] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:49.985249 1260 raft_consensus.cc:2749] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:49.994967 573 leader_election.cc:343] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.003136 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:50.004458 1260 raft_consensus.cc:493] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.005065 1260 raft_consensus.cc:515] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.007176 1260 leader_election.cc:290] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.014781 376 tablet_service.cc:1511] Processing CreateTablet for tablet 338e9b0e89d44295973d4003abd36649 (DEFAULT_TABLE table=default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.016628 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338e9b0e89d44295973d4003abd36649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.018651 872 tablet_service.cc:1511] Processing CreateTablet for tablet 338e9b0e89d44295973d4003abd36649 (DEFAULT_TABLE table=default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.019650 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338e9b0e89d44295973d4003abd36649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.023722 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:50.038283 792 leader_election.cc:343] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.045537 382 tablet_service.cc:1511] Processing CreateTablet for tablet b305a232550e4d4babb19093ea1163f0 (DEFAULT_TABLE table=default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.046689 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b305a232550e4d4babb19093ea1163f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.063771 1265 ts_tablet_manager.cc:1929] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:50.071264 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:50.070716 1265 log.cc:1199] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/6b5eed268ee545ee91420cd792608b23
I20260810 07:52:50.073048 1064 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.074544 871 tablet_service.cc:1511] Processing CreateTablet for tablet b305a232550e4d4babb19093ea1163f0 (DEFAULT_TABLE table=default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.082324 644 tablet_service.cc:1511] Processing CreateTablet for tablet 338e9b0e89d44295973d4003abd36649 (DEFAULT_TABLE table=default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.083027 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b305a232550e4d4babb19093ea1163f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.083988 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338e9b0e89d44295973d4003abd36649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.086596 638 tablet_service.cc:1511] Processing CreateTablet for tablet b305a232550e4d4babb19093ea1163f0 (DEFAULT_TABLE table=default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.090257 412 raft_consensus.cc:3060] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:50.092139 1154 consensus_queue.cc:1048] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:50.094587 32470 master_service.cc:960] ReplaceTablet: received request to replace tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:50.096185 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b305a232550e4d4babb19093ea1163f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:50.099033 792 leader_election.cc:343] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.099597 792 leader_election.cc:304] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.100889 1260 raft_consensus.cc:2749] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:50.110088 1064 tablet_bootstrap.cc:654] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.118670 1265 ts_tablet_manager.cc:1950] T 6b5eed268ee545ee91420cd792608b23 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:50.120523 1300 tablet.cc:1724] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:50.125977 1257 raft_consensus.cc:493] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.126514 377 tablet_service.cc:1511] Processing CreateTablet for tablet 4af5109e338844139444bb7111619195 (DEFAULT_TABLE table=default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.126506 1257 raft_consensus.cc:515] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.114188 32464 catalog_manager.cc:5719] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:50.128038 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af5109e338844139444bb7111619195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.128322 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:50.131264 1154 raft_consensus.cc:493] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.131419 642 tablet_service.cc:1511] Processing CreateTablet for tablet 4af5109e338844139444bb7111619195 (DEFAULT_TABLE table=default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.131840 1154 raft_consensus.cc:515] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.132629 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af5109e338844139444bb7111619195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.136224 1154 leader_election.cc:290] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:50.139511 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fcae5b2e5448bf89b1c0587f46ad40" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:50.141162 573 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.151311 385 tablet_service.cc:1511] Processing CreateTablet for tablet fe43c4aa983843c68a5dcf85201df941 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.152568 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe43c4aa983843c68a5dcf85201df941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.153083 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_536541f9abf14594aaf6e26a1163d266"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:50.154851 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_536541f9abf14594aaf6e26a1163d266 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:50.154808 1257 leader_election.cc:290] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.158638 868 tablet_service.cc:1511] Processing CreateTablet for tablet 4af5109e338844139444bb7111619195 (DEFAULT_TABLE table=default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.159852 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af5109e338844139444bb7111619195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.162442 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_15f51029cf1f4d2a8576e3c2e28ae4a8" } new_table_name: "default.Table_aa23aa2f7ea64c1387df86e35a66e368" modify_external_catalogs: true
I20260810 07:52:50.165511 32460 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7" } modify_external_catalogs: true
I20260810 07:52:50.166095 32460 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7" } modify_external_catalogs: true
I20260810 07:52:50.166622 1154 consensus_queue.cc:1048] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:50.180189 32461 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 4af5109e338844139444bb7111619195
I20260810 07:52:50.191892 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:50.195238 380 tablet_service.cc:1511] Processing CreateTablet for tablet 20a9ab67614949bb93925144c4675f01 (DEFAULT_TABLE table=default.Table_f3fa658df8764c2aa73aa960d170e2a3 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.200379 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a9ab67614949bb93925144c4675f01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:50.205335 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 4af5109e338844139444bb7111619195 because the tablet was not found. No further retry: Not found: Tablet not found: 4af5109e338844139444bb7111619195
I20260810 07:52:50.206959 32467 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a1373ebbd3ba4262806bfc17823d25b5
I20260810 07:52:50.209764 32467 catalog_manager.cc:6807] ReplaceTablet: tablet a1373ebbd3ba4262806bfc17823d25b5 deleted and replaced by tablet 7c075fdf1055477db9f2d56324f12a8a
I20260810 07:52:50.212342 384 tablet_service.cc:1511] Processing CreateTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f (DEFAULT_TABLE table=default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.214592 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c136422ffe4bb2b4f77ab081acd19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.215351 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_247ca9be54cb47c0b9d2f8aee9ac9f3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:50.218545 32461 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 84) took 1033 ms. Trace:
W20260810 07:52:50.222504 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_247ca9be54cb47c0b9d2f8aee9ac9f3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:50.224694 375 tablet_service.cc:1558] Processing DeleteTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c075fdf1055477db9f2d56324f12a8a) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:50.228381 866 tablet_service.cc:1511] Processing CreateTablet for tablet 20a9ab67614949bb93925144c4675f01 (DEFAULT_TABLE table=default.Table_f3fa658df8764c2aa73aa960d170e2a3 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.229477 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:50.229446 646 tablet_service.cc:1511] Processing CreateTablet for tablet fe43c4aa983843c68a5dcf85201df941 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.230208 872 tablet_service.cc:1511] Processing CreateTablet for tablet fe43c4aa983843c68a5dcf85201df941 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.231172 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe43c4aa983843c68a5dcf85201df941. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:50.232072 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet a1373ebbd3ba4262806bfc17823d25b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1373ebbd3ba4262806bfc17823d25b5 already in progress: creating tablet
I20260810 07:52:50.233191 864 tablet_service.cc:1558] Processing DeleteTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c075fdf1055477db9f2d56324f12a8a) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:50.235558 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a9ab67614949bb93925144c4675f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.242960 644 tablet_service.cc:1511] Processing CreateTablet for tablet 20a9ab67614949bb93925144c4675f01 (DEFAULT_TABLE table=default.Table_f3fa658df8764c2aa73aa960d170e2a3 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.243166 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c05d6e49204eedbb305f66dfa2316c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:50.243798 895 raft_consensus.cc:3060] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:50.244069 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a9ab67614949bb93925144c4675f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.245954 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:50.250377 1047 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.250921 1047 ts_tablet_manager.cc:1403] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.602s user 0.014s sys 0.002s
I20260810 07:52:50.253837 1256 mvcc.cc:204] Tried to move back new op lower bound from 7316882921601028096 to 7316882919287603200. Current Snapshot: MvccSnapshot[applied={T|T < 7316882921601028096}]
I20260810 07:52:50.219008 32461 rpcz_store.cc:276] 0810 07:52:49.184985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.188806 (+ 3821us) service_pool.cc:224] Handling call
0810 07:52:49.196105 (+ 7299us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:49.197874 (+ 1769us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:52:49.198028 (+ 154us) catalog_manager.cc:2850] Locking tablets
0810 07:52:49.999930 (+801902us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:52:50.179560 (+179630us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:52:50.179825 (+ 265us) catalog_manager.cc:2898] Committing in-memory state
0810 07:52:50.180011 (+ 186us) catalog_manager.cc:2902] Aborting table tasks
0810 07:52:50.218449 (+ 38438us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:50.025509 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:50.026019 (+ 510us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:50.026051 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:52:50.026067 (+ 16us) tablet.cc:662] Decoding operations
0810 07:52:50.026453 (+ 386us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:50.026504 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:50.026531 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:50.026846 (+ 315us) tablet.cc:701] Row locks acquired
0810 07:52:50.026864 (+ 18us) write_op.cc:260] PREPARE: finished
0810 07:52:50.027006 (+ 142us) write_op.cc:270] Start()
0810 07:52:50.027111 (+ 105us) write_op.cc:276] Timestamp: P: 1786348370026948 usec, L: 0
0810 07:52:50.027132 (+ 21us) op_driver.cc:348] REPLICATION: starting
0810 07:52:50.027740 (+ 608us) log.cc:844] Serialized 730 byte log entry
0810 07:52:50.174772 (+147032us) op_driver.cc:464] REPLICATION: finished
0810 07:52:50.176532 (+ 1760us) write_op.cc:301] APPLY: starting
0810 07:52:50.176597 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:50.176707 (+ 110us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:50.176814 (+ 107us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:50.177109 (+ 295us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:50.177503 (+ 394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:50.177599 (+ 96us) write_op.cc:312] APPLY: finished
0810 07:52:50.177889 (+ 290us) log.cc:844] Serialized 28 byte log entry
0810 07:52:50.178111 (+ 222us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:50.178207 (+ 96us) write_op.cc:454] Released schema lock
0810 07:52:50.178247 (+ 40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1571,"apply.run_cpu_time_us":1981,"apply.run_wall_time_us":1981,"num_ops":2,"prepare.queue_time_us":23126,"prepare.run_cpu_time_us":2744,"prepare.run_wall_time_us":7820,"raft.queue_time_us":251,"raft.run_cpu_time_us":897,"raft.run_wall_time_us":898,"replication_time_us":147522}]]}
I20260810 07:52:50.261555 1265 tablet_replica.cc:333] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:50.262966 1265 raft_consensus.cc:2243] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:50.263767 1265 raft_consensus.cc:2272] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:50.265347 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe43c4aa983843c68a5dcf85201df941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.282675 644 tablet_service.cc:1511] Processing CreateTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f (DEFAULT_TABLE table=default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.286612 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c136422ffe4bb2b4f77ab081acd19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.254361 1047 raft_consensus.cc:359] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.281447 639 tablet_service.cc:1558] Processing DeleteTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c075fdf1055477db9f2d56324f12a8a) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:50.288934 1047 raft_consensus.cc:385] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.289292 1047 raft_consensus.cc:740] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.297086 1047 consensus_queue.cc:260] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.299362 1257 raft_consensus.cc:493] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.299944 1257 raft_consensus.cc:515] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.303030 1257 leader_election.cc:290] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.306176 1265 ts_tablet_manager.cc:1916] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:50.310173 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fcae5b2e5448bf89b1c0587f46ad40" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:50.313995 1047 ts_tablet_manager.cc:1434] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.063s user 0.007s sys 0.001s
I20260810 07:52:50.314419 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:50.315938 1273 consensus_queue.cc:1048] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:50.318313 1302 tablet.cc:1724] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:50.320979 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:50.324577 1064 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.325071 1064 ts_tablet_manager.cc:1403] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.252s user 0.013s sys 0.006s
I20260810 07:52:50.315182 1047 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.348887 895 raft_consensus.cc:2468] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:50.363839 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:50.366053 1047 tablet_bootstrap.cc:654] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.366441 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:50.273054 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_643b0093b7db404e866459a579b26ac0 [id=2e56b185007f4d798dba6e17d0b14818]) successfully deleted
W20260810 07:52:50.371387 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 98370.2us
W20260810 07:52:50.371744 574 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:52:50.371850 385 tablet_service.cc:1511] Processing CreateTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.372918 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d3fcf79a104c47a6fd10fc34f7ce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.372417 574 leader_election.cc:304] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:50.373731 1196 raft_consensus.cc:2749] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:50.374718 864 tablet_service.cc:1511] Processing CreateTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f (DEFAULT_TABLE table=default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.376400 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c136422ffe4bb2b4f77ab081acd19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.377267 574 leader_election.cc:304] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:50.377980 1196 raft_consensus.cc:2804] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:50.378425 1196 raft_consensus.cc:697] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:50.379176 1196 consensus_queue.cc:237] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.332077 1064 raft_consensus.cc:359] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.389132 1064 raft_consensus.cc:385] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.389420 1064 raft_consensus.cc:740] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.129262 32464 catalog_manager.cc:5719] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:50.353289 32470 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:50.391745 32470 catalog_manager.cc:6807] ReplaceTablet: tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 deleted and replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901
I20260810 07:52:50.394307 384 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:50.398794 792 leader_election.cc:343] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:50.399461 792 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:50.400024 792 leader_election.cc:343] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.399655 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:50.400365 792 leader_election.cc:304] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.402354 1064 consensus_queue.cc:260] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.403510 1257 raft_consensus.cc:2749] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:50.403579 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 already in progress: creating tablet
I20260810 07:52:50.407135 869 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:50.408876 32464 catalog_manager.cc:5719] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:50.414960 385 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec00a494816409684205c7f4435a079 (DEFAULT_TABLE table=default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.420627 1047 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.421159 1047 ts_tablet_manager.cc:1403] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.005s
I20260810 07:52:50.424226 1047 raft_consensus.cc:359] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.425968 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec00a494816409684205c7f4435a079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.435915 1064 ts_tablet_manager.cc:1434] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.111s user 0.008s sys 0.002s
I20260810 07:52:50.440878 1047 raft_consensus.cc:385] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.441062 1064 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.441332 1047 raft_consensus.cc:740] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.442224 1047 consensus_queue.cc:260] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:50.442751 792 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.444939 1047 ts_tablet_manager.cc:1434] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260810 07:52:50.449666 1047 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.443225 792 leader_election.cc:304] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.461239 1257 raft_consensus.cc:2749] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:50.470546 872 tablet_service.cc:1511] Processing CreateTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.471467 1047 tablet_bootstrap.cc:654] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.471794 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d3fcf79a104c47a6fd10fc34f7ce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.474634 1331 raft_consensus.cc:493] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.475113 1331 raft_consensus.cc:515] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.475960 1071 tablet_bootstrap.cc:492] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:50.476452 1071 ts_tablet_manager.cc:1403] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.549s user 0.077s sys 0.133s
I20260810 07:52:50.477159 1331 leader_election.cc:290] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:50.480123 309 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:50.479605 1071 raft_consensus.cc:359] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.480749 1071 raft_consensus.cc:385] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.481998 869 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec00a494816409684205c7f4435a079 (DEFAULT_TABLE table=default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.483186 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec00a494816409684205c7f4435a079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.481130 1071 raft_consensus.cc:740] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.492789 1300 tablet.cc:1719] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:50.492501 1071 consensus_queue.cc:260] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.493330 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:50.494524 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]). Expected version 2 got 0
I20260810 07:52:50.497221 1071 ts_tablet_manager.cc:1434] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260810 07:52:50.502228 1071 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:50.510514 574 consensus_peers.cc:597] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e44b480f5f3d4f239eae2586c1d2001a. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:50.511507 1064 tablet_bootstrap.cc:654] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.521763 1071 tablet_bootstrap.cc:654] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.528275 1302 tablet.cc:1719] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:50.529726 1047 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.530153 1047 ts_tablet_manager.cc:1403] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.081s user 0.010s sys 0.000s
I20260810 07:52:50.533416 1047 raft_consensus.cc:359] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.534281 1047 raft_consensus.cc:385] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.534564 1047 raft_consensus.cc:740] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.535255 1047 consensus_queue.cc:260] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.537560 1047 ts_tablet_manager.cc:1434] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 07:52:50.545459 1047 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.551343 1047 tablet_bootstrap.cc:654] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.552702 1064 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.553167 1064 ts_tablet_manager.cc:1403] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.117s user 0.010s sys 0.000s
I20260810 07:52:50.555502 1064 raft_consensus.cc:359] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.556173 1064 raft_consensus.cc:385] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.556416 1064 raft_consensus.cc:740] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.557057 1064 consensus_queue.cc:260] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.469830 1263 ts_tablet_manager.cc:1929] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:50.560194 413 raft_consensus.cc:3060] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:50.560355 1263 log.cc:1199] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/e13d8a5248d54b00aa4a47874561745e
I20260810 07:52:50.561300 1263 ts_tablet_manager.cc:1950] T e13d8a5248d54b00aa4a47874561745e P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:50.569214 1196 consensus_queue.cc:1048] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:50.573323 1064 ts_tablet_manager.cc:1434] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260810 07:52:50.577852 1047 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.578398 1047 ts_tablet_manager.cc:1403] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.003s
I20260810 07:52:50.580760 1047 raft_consensus.cc:359] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.581394 1047 raft_consensus.cc:385] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.581632 1047 raft_consensus.cc:740] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.581660 644 tablet_service.cc:1511] Processing CreateTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.582183 1047 consensus_queue.cc:260] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.582760 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9d3fcf79a104c47a6fd10fc34f7ce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.584071 1047 ts_tablet_manager.cc:1434] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:52:50.584895 1047 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.596698 1047 tablet_bootstrap.cc:654] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.603547 1064 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.613523 639 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:50.615368 639 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec00a494816409684205c7f4435a079 (DEFAULT_TABLE table=default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:50.615949 1064 tablet_bootstrap.cc:654] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.616845 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec00a494816409684205c7f4435a079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:50.618927 1257 raft_consensus.cc:493] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.619411 1257 raft_consensus.cc:515] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.621330 1257 leader_election.cc:290] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.622885 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1bcc922db45ad9f5f0edac6c7b5d5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:50.624773 792 leader_election.cc:343] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.629681 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a1bcc922db45ad9f5f0edac6c7b5d5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:50.631364 792 leader_election.cc:343] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.631798 792 leader_election.cc:304] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.634380 1257 raft_consensus.cc:2749] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:50.634527 1353 log.cc:927] Time spent T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.134s user 0.001s sys 0.000s
W20260810 07:52:50.505178 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9369baf30ab64646b7bc042c6da1987b consensus state has no leader
W20260810 07:52:50.541410 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260810 07:52:50.654428 1047 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.654863 1047 ts_tablet_manager.cc:1403] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.070s user 0.014s sys 0.003s
I20260810 07:52:50.658047 1064 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.658456 1064 ts_tablet_manager.cc:1403] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.013s
I20260810 07:52:50.661314 1047 raft_consensus.cc:359] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.661979 1047 raft_consensus.cc:385] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.662245 1047 raft_consensus.cc:740] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:52:50.663198 308 leader_election.cc:343] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:50.663650 308 leader_election.cc:304] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.664673 1331 raft_consensus.cc:2749] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:50.664803 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 123407us
I20260810 07:52:50.665534 1047 consensus_queue.cc:260] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.666971 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 43) took 1718 ms. Trace:
I20260810 07:52:50.667730 1047 ts_tablet_manager.cc:1434] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260810 07:52:50.667311 1263 rpcz_store.cc:276] 0810 07:52:48.948111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:48.948275 (+ 164us) service_pool.cc:224] Handling call
0810 07:52:49.351305 (+403030us) ts_tablet_manager.cc:1151] Deleting tablet e13d8a5248d54b00aa4a47874561745e
0810 07:52:49.351346 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:49.660616 (+309270us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:50.469684 (+809068us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:50.666909 (+197225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":402329}
I20260810 07:52:50.668076 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 4ec00a494816409684205c7f4435a079
I20260810 07:52:50.660696 1064 raft_consensus.cc:359] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.669402 1064 raft_consensus.cc:385] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.669736 1064 raft_consensus.cc:740] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.668612 1047 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.674475 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:50.675267 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_5152115799f74951b88c4268d7aeeec9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:50.675843 1047 tablet_bootstrap.cc:654] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:50.677527 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 4ec00a494816409684205c7f4435a079 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ec00a494816409684205c7f4435a079 already in progress: creating tablet
I20260810 07:52:50.677222 1064 consensus_queue.cc:260] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.679425 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:50.683043 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_64987f4480ff416b8bd9688ac5df15a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:50.684360 1064 ts_tablet_manager.cc:1434] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.026s user 0.002s sys 0.005s
I20260810 07:52:50.685272 1064 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.685505 642 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:50.686676 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_64987f4480ff416b8bd9688ac5df15a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:50.668622 1263 tablet_replica.cc:333] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:50.688623 1263 raft_consensus.cc:2243] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
W20260810 07:52:50.689517 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5152115799f74951b88c4268d7aeeec9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:50.689623 1263 raft_consensus.cc:2272] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:50.691133 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet e13d8a5248d54b00aa4a47874561745e (table default.Table_753b8ca312194a6d9e028c358cbe2007 [id=9e498c00e46644838f7f0be959e831b0]) successfully deleted
I20260810 07:52:50.698022 1263 ts_tablet_manager.cc:1916] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:50.704489 1064 tablet_bootstrap.cc:654] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:50.705435 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 92e2678ab165486dbbb69c7c183a7fa4 has no consensus state
I20260810 07:52:50.706405 32469 master_service.cc:960] ReplaceTablet: received request to replace tablet c3675c5bae7a42769223ef5b2fc433b3 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:50.714740 1265 ts_tablet_manager.cc:1929] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:50.715252 1265 log.cc:1199] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:50.716089 1265 ts_tablet_manager.cc:1950] T 95fcae5b2e5448bf89b1c0587f46ad40 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:50.717962 1064 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.718353 1064 ts_tablet_manager.cc:1403] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.004s
I20260810 07:52:50.718905 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 95fcae5b2e5448bf89b1c0587f46ad40 (table default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]) successfully deleted
I20260810 07:52:50.722625 1071 tablet_bootstrap.cc:492] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:50.728444 1064 raft_consensus.cc:359] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.729151 1064 raft_consensus.cc:385] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.729445 1064 raft_consensus.cc:740] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.730114 1064 consensus_queue.cc:260] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.734251 1071 ts_tablet_manager.cc:1403] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.232s user 0.039s sys 0.015s
I20260810 07:52:50.739197 1263 ts_tablet_manager.cc:1929] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:50.739593 1263 log.cc:1199] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/0b4772c3359a426e9f105760afff40cb
I20260810 07:52:50.739732 1071 raft_consensus.cc:359] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.740384 1263 ts_tablet_manager.cc:1950] T 0b4772c3359a426e9f105760afff40cb P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:50.740453 1071 raft_consensus.cc:385] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.741040 1071 raft_consensus.cc:740] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.744149 1064 ts_tablet_manager.cc:1434] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260810 07:52:50.742071 1071 consensus_queue.cc:260] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.746230 1071 ts_tablet_manager.cc:1434] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260810 07:52:50.749689 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 4af5109e338844139444bb7111619195 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af5109e338844139444bb7111619195 already in progress: creating tablet
I20260810 07:52:50.745009 1064 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.752365 1265 tablet_replica.cc:333] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:50.755744 1265 raft_consensus.cc:2243] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:50.756275 1265 raft_consensus.cc:2272] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:50.757257 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260810 07:52:50.758303 1265 ts_tablet_manager.cc:1916] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:50.759580 1064 tablet_bootstrap.cc:654] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.762312 1071 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:50.764236 1257 raft_consensus.cc:493] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.764665 1257 raft_consensus.cc:515] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.766515 1257 leader_election.cc:290] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.768566 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec00a494816409684205c7f4435a079" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:50.770517 792 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.772370 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec00a494816409684205c7f4435a079" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:50.774294 792 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.774744 792 leader_election.cc:304] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:50.775816 1257 raft_consensus.cc:2749] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:50.777168 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 44) took 1816 ms. Trace:
I20260810 07:52:50.779951 1263 rpcz_store.cc:276] 0810 07:52:48.961202 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:48.961346 (+ 144us) service_pool.cc:224] Handling call
0810 07:52:50.668468 (+1707122us) ts_tablet_manager.cc:1151] Deleting tablet 0b4772c3359a426e9f105760afff40cb
0810 07:52:50.668512 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:50.723423 (+ 54911us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:50.739064 (+ 15641us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:50.777093 (+ 38029us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1706472}
I20260810 07:52:50.781433 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 0b4772c3359a426e9f105760afff40cb (table default.Table_39969fe570e440f2a2edc5187b76e63b [id=f652987efd2846e08aff630aa3a20f14]) successfully deleted
I20260810 07:52:50.786164 1071 tablet_bootstrap.cc:654] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.787791 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_643b0093b7db404e866459a579b26ac0" } modify_external_catalogs: true
I20260810 07:52:50.788414 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_643b0093b7db404e866459a579b26ac0" } modify_external_catalogs: true
I20260810 07:52:50.792403 1047 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.792902 1047 ts_tablet_manager.cc:1403] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.124s user 0.010s sys 0.017s
I20260810 07:52:50.795701 1047 raft_consensus.cc:359] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.796335 1047 raft_consensus.cc:385] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.796579 1047 raft_consensus.cc:740] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.798961 1263 tablet_replica.cc:333] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:50.799834 1263 raft_consensus.cc:2243] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:50.800387 1263 raft_consensus.cc:2272] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:50.801326 1047 consensus_queue.cc:260] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.804977 1263 ts_tablet_manager.cc:1916] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:50.811234 1071 tablet_bootstrap.cc:492] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:50.811662 1071 ts_tablet_manager.cc:1403] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.050s user 0.005s sys 0.005s
I20260810 07:52:50.813473 1047 ts_tablet_manager.cc:1434] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260810 07:52:50.814373 1047 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:50.814215 1071 raft_consensus.cc:359] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.825839 1071 raft_consensus.cc:385] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.826143 1071 raft_consensus.cc:740] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.826753 1071 consensus_queue.cc:260] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.829393 1071 ts_tablet_manager.cc:1434] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260810 07:52:50.833829 1071 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:50.836437 32465 master_service.cc:960] ReplaceTablet: received request to replace tablet 3f679d04625b4c0690a4ce8ea2a5e87b from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:50.844607 1047 tablet_bootstrap.cc:654] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.846838 1265 ts_tablet_manager.cc:1929] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:50.847002 1064 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.847213 1265 log.cc:1199] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/a1373ebbd3ba4262806bfc17823d25b5
I20260810 07:52:50.847383 1064 ts_tablet_manager.cc:1403] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.103s user 0.007s sys 0.006s
I20260810 07:52:50.847865 1265 ts_tablet_manager.cc:1950] T a1373ebbd3ba4262806bfc17823d25b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:50.850314 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet a1373ebbd3ba4262806bfc17823d25b5 (table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]) successfully deleted
I20260810 07:52:50.850270 1064 raft_consensus.cc:359] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.850914 1064 raft_consensus.cc:385] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.851193 1064 raft_consensus.cc:740] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.851919 1064 consensus_queue.cc:260] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.857143 1257 raft_consensus.cc:493] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.857575 1257 raft_consensus.cc:515] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.859516 1257 leader_election.cc:290] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:50.861138 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:50.862815 792 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:50.863653 1265 tablet_replica.cc:333] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:50.865007 1071 tablet_bootstrap.cc:654] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.867738 1064 ts_tablet_manager.cc:1434] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
I20260810 07:52:50.868402 1265 raft_consensus.cc:2243] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:50.869004 1265 raft_consensus.cc:2272] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:50.874563 1265 ts_tablet_manager.cc:1916] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:50.886379 1047 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:50.886821 1047 ts_tablet_manager.cc:1403] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.073s user 0.009s sys 0.000s
I20260810 07:52:50.887210 32460 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0f6aa0ec81114d78abb2ad53a02bf930" } modify_external_catalogs: true
I20260810 07:52:50.887898 32460 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0f6aa0ec81114d78abb2ad53a02bf930" } modify_external_catalogs: true
I20260810 07:52:50.889403 1047 raft_consensus.cc:359] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.890069 1047 raft_consensus.cc:385] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.879508 1064 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:50.900112 1263 ts_tablet_manager.cc:1929] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:50.900550 1263 log.cc:1199] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/6b5eed268ee545ee91420cd792608b23
I20260810 07:52:50.902634 1064 tablet_bootstrap.cc:654] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.905676 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:50.907358 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:50.909430 32470 catalog_manager.cc:5719] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:52:50.910297 32469 catalog_manager.cc:6807] ReplaceTablet: tablet c3675c5bae7a42769223ef5b2fc433b3 deleted and replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:52:50.910750 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:50.911295 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:50.914177 866 tablet_service.cc:1558] Processing DeleteTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:50.919559 376 tablet_service.cc:1558] Processing DeleteTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:50.920083 1071 tablet_bootstrap.cc:492] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:50.920634 1071 ts_tablet_manager.cc:1403] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.000s
W20260810 07:52:50.921577 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet c3675c5bae7a42769223ef5b2fc433b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3675c5bae7a42769223ef5b2fc433b3 already in progress: creating tablet
I20260810 07:52:50.924007 1071 raft_consensus.cc:359] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.924973 1071 raft_consensus.cc:385] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:50.925299 1071 raft_consensus.cc:740] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.925514 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d6e2c1a2cec54b099ba095449ee1b5a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:50.926244 1071 consensus_queue.cc:260] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:50.927968 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6e2c1a2cec54b099ba095449ee1b5a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:50.928422 1071 ts_tablet_manager.cc:1434] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260810 07:52:50.944005 1071 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:50.955821 1265 ts_tablet_manager.cc:1929] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:50.956389 1265 log.cc:1199] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:50.969388 1265 ts_tablet_manager.cc:1950] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:50.972886 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]) successfully deleted
I20260810 07:52:50.978333 1071 tablet_bootstrap.cc:654] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:50.979696 1064 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:50.980134 1064 ts_tablet_manager.cc:1403] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.101s user 0.036s sys 0.031s
I20260810 07:52:50.890333 1047 raft_consensus.cc:740] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:50.983718 1047 consensus_queue.cc:260] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.986356 1154 raft_consensus.cc:493] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.987403 1154 raft_consensus.cc:515] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.988355 1196 raft_consensus.cc:493] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.989108 1273 raft_consensus.cc:493] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.989068 1196 raft_consensus.cc:515] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.989756 1273 raft_consensus.cc:515] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:50.991815 1273 leader_election.cc:290] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:50.993512 1263 ts_tablet_manager.cc:1950] T 6b5eed268ee545ee91420cd792608b23 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:50.994779 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:50.995401 895 raft_consensus.cc:2468] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:50.996183 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:50.996829 415 raft_consensus.cc:2468] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:50.996876 1331 raft_consensus.cc:493] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:50.997411 1331 raft_consensus.cc:515] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:50.998375 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:50.999135 896 raft_consensus.cc:2468] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:50.999528 1331 leader_election.cc:290] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.000289 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:51.004173 1154 leader_election.cc:290] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.004539 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:51.005364 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 50) took 1064 ms. Trace:
I20260810 07:52:51.005883 1263 rpcz_store.cc:276] 0810 07:52:49.941126 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.941832 (+ 706us) service_pool.cc:224] Handling call
0810 07:52:50.798797 (+856965us) ts_tablet_manager.cc:1151] Deleting tablet 6b5eed268ee545ee91420cd792608b23
0810 07:52:50.798843 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:50.827855 (+ 29012us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:50.899955 (+ 72100us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.005195 (+105240us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":855987}
I20260810 07:52:51.008172 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:51.009016 1263 tablet_replica.cc:333] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:51.011382 1265 tablet_replica.cc:333] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:51.012167 1265 raft_consensus.cc:2243] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.006459 573 leader_election.cc:304] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:51.012364 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:51.013124 896 raft_consensus.cc:2468] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:52:51.013446 573 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.013644 1154 raft_consensus.cc:2804] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.014008 1154 raft_consensus.cc:493] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.014349 1154 raft_consensus.cc:3060] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.015863 574 leader_election.cc:304] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.016896 1273 raft_consensus.cc:2804] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.017241 1273 raft_consensus.cc:493] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.017326 1196 leader_election.cc:290] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.017587 1273 raft_consensus.cc:3060] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:51.014351 573 leader_election.cc:343] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.024686 574 leader_election.cc:304] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.012606 1265 raft_consensus.cc:2272] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.025694 1196 raft_consensus.cc:2804] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.026080 1196 raft_consensus.cc:493] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.026399 1196 raft_consensus.cc:3060] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.028172 1265 ts_tablet_manager.cc:1916] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.031133 1154 raft_consensus.cc:515] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.042765 1071 tablet_bootstrap.cc:492] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:51.043408 1071 ts_tablet_manager.cc:1403] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.006s
I20260810 07:52:51.046324 1071 raft_consensus.cc:359] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.034703 1047 ts_tablet_manager.cc:1434] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.148s user 0.010s sys 0.000s
I20260810 07:52:51.048527 1047 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:51.050009 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:51.050691 412 raft_consensus.cc:3060] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.046314 1263 raft_consensus.cc:2243] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.052315 1263 raft_consensus.cc:2272] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.054648 1263 ts_tablet_manager.cc:1916] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.056030 1257 raft_consensus.cc:493] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.056506 1257 raft_consensus.cc:515] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.058284 412 raft_consensus.cc:2468] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.058604 1257 leader_election.cc:290] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.059818 573 leader_election.cc:304] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:51.061074 1390 raft_consensus.cc:2804] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.061000 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af5109e338844139444bb7111619195" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:51.061721 1390 raft_consensus.cc:697] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
W20260810 07:52:51.065086 792 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.046753 1064 raft_consensus.cc:359] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.066324 1064 raft_consensus.cc:385] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.066620 1064 raft_consensus.cc:740] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.066926 1390 consensus_queue.cc:237] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.069439 1154 leader_election.cc:290] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.069896 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 3f679d04625b4c0690a4ce8ea2a5e87b
I20260810 07:52:51.070125 1071 raft_consensus.cc:385] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.070407 1071 raft_consensus.cc:740] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.071432 32465 catalog_manager.cc:6807] ReplaceTablet: tablet 3f679d04625b4c0690a4ce8ea2a5e87b deleted and replaced by tablet b6226a2377864d47b14901df39677127
I20260810 07:52:51.071053 1071 consensus_queue.cc:260] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.074332 866 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b with delete_type TABLET_DATA_DELETED (replaced by tablet b6226a2377864d47b14901df39677127) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.075106 1064 consensus_queue.cc:260] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.076107 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b with delete_type TABLET_DATA_DELETED (replaced by tablet b6226a2377864d47b14901df39677127) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:51.076892 1103 tablet_replica.cc:333] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:51.077668 1103 raft_consensus.cc:2243] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.080099 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9920a33cc93419ca3284a0b009236df" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:51.080667 896 raft_consensus.cc:3060] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.084898 1103 raft_consensus.cc:2272] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.086791 1103 ts_tablet_manager.cc:1916] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.090104 896 raft_consensus.cc:2468] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.102008 1071 ts_tablet_manager.cc:1434] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.058s user 0.005s sys 0.001s
I20260810 07:52:51.118516 1064 ts_tablet_manager.cc:1434] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.138s user 0.002s sys 0.005s
I20260810 07:52:51.118726 1273 raft_consensus.cc:515] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.119565 1064 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:51.122692 1196 raft_consensus.cc:515] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.124368 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 6b5eed268ee545ee91420cd792608b23 (table default.Table_643b0093b7db404e866459a579b26ac0 [id=2e56b185007f4d798dba6e17d0b14818]) successfully deleted
I20260810 07:52:51.128134 1047 tablet_bootstrap.cc:654] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.131315 1196 leader_election.cc:290] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.133507 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:51.134207 895 raft_consensus.cc:3060] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.135138 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f353e1e07485400a964f8e05b0691cc6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:51.138108 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:51.139755 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af5109e338844139444bb7111619195" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:51.139943 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1eb27f536648c888a5acc3e940e4c0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:51.140641 412 raft_consensus.cc:3060] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:51.141042 792 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.141393 895 raft_consensus.cc:2468] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.141533 792 leader_election.cc:304] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:51.142375 1257 raft_consensus.cc:2749] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:51.143522 792 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.143569 574 leader_election.cc:304] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:51.143952 792 leader_election.cc:304] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:51.144871 1257 raft_consensus.cc:2749] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:51.144507 1154 raft_consensus.cc:2804] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.146119 1154 raft_consensus.cc:697] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:51.147122 1154 consensus_queue.cc:237] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.156653 1071 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:51.163376 1064 tablet_bootstrap.cc:654] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.167275 415 raft_consensus.cc:3060] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.174224 412 raft_consensus.cc:2468] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.181881 1265 ts_tablet_manager.cc:1929] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:51.182396 1196 raft_consensus.cc:493] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.182938 1196 raft_consensus.cc:515] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.190026 1047 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:51.190547 1047 ts_tablet_manager.cc:1403] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.142s user 0.008s sys 0.003s
I20260810 07:52:51.191871 1103 ts_tablet_manager.cc:1929] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:51.192334 1103 log.cc:1199] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/3f679d04625b4c0690a4ce8ea2a5e87b
I20260810 07:52:51.193559 1103 ts_tablet_manager.cc:1950] T 3f679d04625b4c0690a4ce8ea2a5e87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:51.197474 1047 raft_consensus.cc:359] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.198140 1047 raft_consensus.cc:385] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.198398 1047 raft_consensus.cc:740] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.194212 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:51.199498 1047 consensus_queue.cc:260] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.182642 1257 raft_consensus.cc:493] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:52:51.200289 573 leader_election.cc:343] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.200409 1257 raft_consensus.cc:515] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.202487 1257 leader_election.cc:290] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.202924 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:51.203522 896 raft_consensus.cc:2468] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:51.203637 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_89a4977bd2c24950a1cb9133ad6a157e" } new_table_name: "default.Table_7e58a302aba14a68a5c4281837dbb971" modify_external_catalogs: true
I20260810 07:52:51.204845 574 leader_election.cc:304] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.205735 1196 leader_election.cc:290] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.205938 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:51.206746 1196 raft_consensus.cc:2804] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.207118 1196 raft_consensus.cc:493] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.207463 1196 raft_consensus.cc:3060] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.207993 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:51.209455 792 leader_election.cc:343] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.210763 1263 ts_tablet_manager.cc:1929] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:51.211205 1263 log.cc:1199] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:51.211262 1265 log.cc:1199] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/4ec00a494816409684205c7f4435a079
I20260810 07:52:51.212085 1265 ts_tablet_manager.cc:1950] T 4ec00a494816409684205c7f4435a079 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:51.212087 1263 ts_tablet_manager.cc:1950] T 95fcae5b2e5448bf89b1c0587f46ad40 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:52:51.212518 792 leader_election.cc:343] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.213330 792 leader_election.cc:304] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:51.213616 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_89a4977bd2c24950a1cb9133ad6a157e [id=57fbc9f2436f42289d66a57eec7232a9]) successfully deleted
I20260810 07:52:51.214329 1257 raft_consensus.cc:2749] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:51.217278 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 4ec00a494816409684205c7f4435a079 (table default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]) successfully deleted
I20260810 07:52:51.223945 1265 tablet_replica.cc:333] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:51.224534 1047 ts_tablet_manager.cc:1434] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.034s user 0.008s sys 0.000s
I20260810 07:52:51.224953 1265 raft_consensus.cc:2243] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.225453 1047 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:51.225471 1265 raft_consensus.cc:2272] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.227246 1265 ts_tablet_manager.cc:1916] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.230427 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 51) took 1288 ms. Trace:
I20260810 07:52:51.230813 1263 rpcz_store.cc:276] 0810 07:52:49.941594 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.942958 (+ 1364us) service_pool.cc:224] Handling call
0810 07:52:51.008666 (+1065708us) ts_tablet_manager.cc:1151] Deleting tablet 95fcae5b2e5448bf89b1c0587f46ad40
0810 07:52:51.008820 (+ 154us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:51.095219 (+ 86399us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.210635 (+115416us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.230353 (+ 19718us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35801,"tablet-delete.queue_time_us":1065040}
I20260810 07:52:51.232787 639 tablet_service.cc:1558] Processing DeleteTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.233863 1064 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:51.234314 1064 ts_tablet_manager.cc:1403] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.000s
I20260810 07:52:51.235280 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f679d04625b4c0690a4ce8ea2a5e87b with delete_type TABLET_DATA_DELETED (replaced by tablet b6226a2377864d47b14901df39677127) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.237195 308 leader_election.cc:304] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:51.238336 1064 raft_consensus.cc:359] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.238445 1196 raft_consensus.cc:515] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.239231 1064 raft_consensus.cc:385] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.239557 1064 raft_consensus.cc:740] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.241663 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:51.242372 896 raft_consensus.cc:3060] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.244002 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:51.245476 573 leader_election.cc:343] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.249800 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:51.250191 1273 leader_election.cc:290] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.250353 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3675c5bae7a42769223ef5b2fc433b3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:52:51.252005 574 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:52:51.252849 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 95fcae5b2e5448bf89b1c0587f46ad40 (table default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]) successfully deleted
W20260810 07:52:51.252974 573 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.253377 573 leader_election.cc:304] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:51.259120 896 raft_consensus.cc:2468] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.261065 574 leader_election.cc:304] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.261551 1071 tablet_bootstrap.cc:654] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.261916 1390 raft_consensus.cc:2749] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:51.262547 1390 raft_consensus.cc:2804] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.262959 1390 raft_consensus.cc:697] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:51.263782 1390 consensus_queue.cc:237] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.269162 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 56) took 1028 ms. Trace:
I20260810 07:52:51.269595 1263 rpcz_store.cc:276] 0810 07:52:50.241319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.245713 (+ 4394us) service_pool.cc:224] Handling call
0810 07:52:51.231922 (+986209us) ts_tablet_manager.cc:1151] Deleting tablet 4af5109e338844139444bb7111619195
0810 07:52:51.252513 (+ 20591us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:51.269064 (+ 16551us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":985427}
I20260810 07:52:51.271853 1263 tablet_replica.cc:333] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:52:51.274255 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 4af5109e338844139444bb7111619195 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af5109e338844139444bb7111619195 already in progress: creating tablet
I20260810 07:52:51.277980 1196 leader_election.cc:290] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.240406 1064 consensus_queue.cc:260] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.283324 1263 raft_consensus.cc:2243] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.285264 1071 tablet_bootstrap.cc:492] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:51.285518 1263 raft_consensus.cc:2272] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.285722 1071 ts_tablet_manager.cc:1403] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.129s user 0.006s sys 0.005s
I20260810 07:52:51.287827 1263 ts_tablet_manager.cc:1916] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.297158 1331 raft_consensus.cc:2749] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:51.303536 1064 ts_tablet_manager.cc:1434] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.069s user 0.008s sys 0.000s
I20260810 07:52:51.320885 1047 tablet_bootstrap.cc:654] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.322818 1265 ts_tablet_manager.cc:1929] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:51.326961 1196 raft_consensus.cc:493] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.327486 1196 raft_consensus.cc:515] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.328513 1273 consensus_queue.cc:1048] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:51.331871 1196 leader_election.cc:290] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.333853 1408 tablet.cc:1724] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.339341 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:51.341391 573 leader_election.cc:343] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.342842 1071 raft_consensus.cc:359] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.347296 1071 raft_consensus.cc:385] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.349063 1071 raft_consensus.cc:740] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.354139 1071 consensus_queue.cc:260] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.328330 1265 log.cc:1199] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:51.367928 1265 ts_tablet_manager.cc:1950] T c3675c5bae7a42769223ef5b2fc433b3 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:51.370635 1071 ts_tablet_manager.cc:1434] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.085s user 0.005s sys 0.005s
I20260810 07:52:51.373003 1263 ts_tablet_manager.cc:1929] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:51.373416 1263 log.cc:1199] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/a1373ebbd3ba4262806bfc17823d25b5
I20260810 07:52:51.374146 1263 ts_tablet_manager.cc:1950] T a1373ebbd3ba4262806bfc17823d25b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:52:51.375911 573 consensus_peers.cc:597] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f679d04625b4c0690a4ce8ea2a5e87b. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:51.333457 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:51.386368 895 raft_consensus.cc:2468] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:51.387652 574 leader_election.cc:304] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.329448 1064 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:51.390260 415 raft_consensus.cc:3060] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.393060 1071 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:51.397761 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 58) took 1155 ms. Trace:
I20260810 07:52:51.398435 1390 raft_consensus.cc:2804] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.398960 1390 raft_consensus.cc:493] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.399353 1390 raft_consensus.cc:3060] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.401834 1196 raft_consensus.cc:493] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.402302 1196 raft_consensus.cc:515] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.404366 1196 leader_election.cc:290] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.406248 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:51.407729 573 leader_election.cc:343] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.407756 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:51.398192 1263 rpcz_store.cc:276] 0810 07:52:50.242473 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.281201 (+ 38728us) service_pool.cc:224] Handling call
0810 07:52:51.271610 (+990409us) ts_tablet_manager.cc:1151] Deleting tablet a1373ebbd3ba4262806bfc17823d25b5
0810 07:52:51.271690 (+ 80us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:51.312353 (+ 40663us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.372850 (+ 60497us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.397681 (+ 24831us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":982474}
I20260810 07:52:51.408393 896 raft_consensus.cc:2468] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:51.409159 1263 tablet_replica.cc:333] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:51.410027 1263 raft_consensus.cc:2243] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.410521 1064 tablet_bootstrap.cc:654] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.410423 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet a1373ebbd3ba4262806bfc17823d25b5 (table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]) successfully deleted
I20260810 07:52:51.410547 1263 raft_consensus.cc:2272] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.410954 574 leader_election.cc:304] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.411693 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:51.411900 1273 raft_consensus.cc:2804] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.412160 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:51.412212 1273 raft_consensus.cc:493] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:51.412606 1273 raft_consensus.cc:3060] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.418310 1390 raft_consensus.cc:515] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.421492 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:51.421605 1273 raft_consensus.cc:515] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:51.422984 573 leader_election.cc:343] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.424585 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:51.425196 896 raft_consensus.cc:3060] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.425390 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:51.426723 573 leader_election.cc:343] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.432155 896 raft_consensus.cc:2468] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.412638 1263 ts_tablet_manager.cc:1916] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.438239 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet c3675c5bae7a42769223ef5b2fc433b3 (table default.Table_3a887c70a4ef45ccace031aa6fa907de [id=43a67562d28b442a86908e5e13f18832]) successfully deleted
I20260810 07:52:51.439216 1265 tablet_replica.cc:333] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:51.439185 1273 leader_election.cc:290] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.440035 1265 raft_consensus.cc:2243] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.440529 1265 raft_consensus.cc:2272] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:51.440696 574 leader_election.cc:304] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.442373 1265 ts_tablet_manager.cc:1916] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:51.443259 1273 raft_consensus.cc:2804] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.444587 376 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 (DEFAULT_TABLE table=default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.445916 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4b3ca46c9c44559761e25bcde3e927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.446962 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:51.447867 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:51.446691 1273 raft_consensus.cc:697] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:51.449208 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:51.449245 1273 consensus_queue.cc:237] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.450321 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]). Expected version 2 got 0
W20260810 07:52:51.451225 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6226a2377864d47b14901df39677127 has no consensus state
I20260810 07:52:51.458652 644 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 (DEFAULT_TABLE table=default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.458911 866 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 (DEFAULT_TABLE table=default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.459898 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4b3ca46c9c44559761e25bcde3e927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.471259 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:51.471709 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:51.476686 1064 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:51.479043 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]). Expected version 1 got 0
I20260810 07:52:51.480090 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:51.480865 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:51.481585 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 1 got 0
I20260810 07:52:51.482290 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:51.483227 32467 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet e69ea1aae7784e75b7a7b3f5d67c1658
I20260810 07:52:51.483282 1196 consensus_queue.cc:1048] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:51.490983 1408 tablet.cc:1719] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:51.495428 1071 tablet_bootstrap.cc:654] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.495370 1390 leader_election.cc:290] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:51.499889 639 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 (DEFAULT_TABLE table=default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.501121 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae4a28e89714c65b7551f36d1dc78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.501737 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 1 got 0
I20260810 07:52:51.501880 382 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 (DEFAULT_TABLE table=default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.502605 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:51.502887 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae4a28e89714c65b7551f36d1dc78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.503049 1064 ts_tablet_manager.cc:1403] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.174s user 0.012s sys 0.000s
I20260810 07:52:51.503178 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:51.503811 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:51.504709 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]). Expected version 1 got 0
I20260810 07:52:51.505550 1064 raft_consensus.cc:359] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.506175 1064 raft_consensus.cc:385] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.506438 1064 raft_consensus.cc:740] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.496877 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:51.507211 895 raft_consensus.cc:3060] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.507107 1064 consensus_queue.cc:260] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:51.510620 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6226a2377864d47b14901df39677127 has no consensus state
I20260810 07:52:51.512207 32469 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:51.513204 32469 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:51.513803 32469 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:51.514567 32469 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:51.521443 1408 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.521754 872 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 (DEFAULT_TABLE table=default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.520404 895 raft_consensus.cc:2468] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:51.523685 574 leader_election.cc:304] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.525367 1273 raft_consensus.cc:2804] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.525434 385 tablet_service.cc:1511] Processing CreateTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.526556 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e2678ab165486dbbb69c7c183a7fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.526499 1273 raft_consensus.cc:697] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:51.527333 1273 consensus_queue.cc:237] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.530233 1430 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.532974 1047 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:51.533365 1047 ts_tablet_manager.cc:1403] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.308s user 0.013s sys 0.014s
I20260810 07:52:51.534451 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:51.535755 1047 raft_consensus.cc:359] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.536386 1047 raft_consensus.cc:385] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.537021 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d]). Expected version 2 got 0
I20260810 07:52:51.539198 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_aeb2755859d043bfa96d14a40dddb4be [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 1 got 0
I20260810 07:52:51.546283 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4b3ca46c9c44559761e25bcde3e927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.547022 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae4a28e89714c65b7551f36d1dc78b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.536640 1047 raft_consensus.cc:740] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.552577 1047 consensus_queue.cc:260] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.570269 1047 ts_tablet_manager.cc:1434] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260810 07:52:51.580765 644 tablet_service.cc:1511] Processing CreateTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.584161 864 tablet_service.cc:1511] Processing CreateTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.585421 1257 raft_consensus.cc:493] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.586410 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e2678ab165486dbbb69c7c183a7fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.586215 1257 raft_consensus.cc:515] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.587359 1047 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:51.595182 1257 leader_election.cc:290] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.597857 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:51.599080 676 raft_consensus.cc:2468] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:51.601507 792 leader_election.cc:304] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:51.601612 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:51.602449 1257 raft_consensus.cc:2804] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:51.603101 1257 raft_consensus.cc:493] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260810 07:52:51.603457 792 leader_election.cc:343] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.603673 1257 raft_consensus.cc:3060] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.607508 1431 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.615218 376 tablet_service.cc:1511] Processing CreateTablet for tablet c1826ebc26f04805b70ac5314e48fda9 (DEFAULT_TABLE table=default.Table_d73cd38f1d774b448a86e7e8f5b10242 [id=e11987dccb6140aaa52a5ad9d2f5e2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.586261 1064 ts_tablet_manager.cc:1434] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.083s user 0.005s sys 0.000s
I20260810 07:52:51.616705 1064 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:51.615813 32467 catalog_manager.cc:5719] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:51.617527 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]). Expected version 2 got 0
I20260810 07:52:51.618424 32467 catalog_manager.cc:5719] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:51.619259 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1826ebc26f04805b70ac5314e48fda9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.631982 1071 tablet_bootstrap.cc:492] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:51.632827 1071 ts_tablet_manager.cc:1403] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.241s user 0.017s sys 0.004s
I20260810 07:52:51.636649 1071 raft_consensus.cc:359] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.641937 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e2678ab165486dbbb69c7c183a7fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.653146 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:51.653673 1071 raft_consensus.cc:385] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.654633 32467 catalog_manager.cc:5719] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:51.655957 646 tablet_service.cc:1511] Processing CreateTablet for tablet c1826ebc26f04805b70ac5314e48fda9 (DEFAULT_TABLE table=default.Table_d73cd38f1d774b448a86e7e8f5b10242 [id=e11987dccb6140aaa52a5ad9d2f5e2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.657434 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1826ebc26f04805b70ac5314e48fda9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.664749 1071 raft_consensus.cc:740] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.665750 1071 consensus_queue.cc:260] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.671648 866 tablet_service.cc:1511] Processing CreateTablet for tablet c1826ebc26f04805b70ac5314e48fda9 (DEFAULT_TABLE table=default.Table_d73cd38f1d774b448a86e7e8f5b10242 [id=e11987dccb6140aaa52a5ad9d2f5e2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.675422 1071 ts_tablet_manager.cc:1434] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260810 07:52:51.676323 1071 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:51.679333 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1826ebc26f04805b70ac5314e48fda9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.685264 385 tablet_service.cc:1511] Processing CreateTablet for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (DEFAULT_TABLE table=default.Table_680195385fd2411baf85a6a49bab6feb [id=ee979df2f5bc4651a69dfee31d80590f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.686503 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.669353 32467 catalog_manager.cc:5719] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:51.687402 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_395605094533475e8e04175ec53c96bb [id=a49a08c7a7954b87aab561894073bf0d]). Expected version 1 got 0
I20260810 07:52:51.688151 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 1 got 0
I20260810 07:52:51.690896 32463 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 78) took 2510 ms. Trace:
I20260810 07:52:51.693095 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet f353e1e07485400a964f8e05b0691cc6
W20260810 07:52:51.698066 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 25) took 2442 ms (client timeout 1000 ms). Trace:
W20260810 07:52:51.698576 32462 rpcz_store.cc:269] 0810 07:52:49.255480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.297156 (+ 41676us) service_pool.cc:224] Handling call
0810 07:52:49.326432 (+ 29276us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:51.697961 (+2371529us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":64583552}
I20260810 07:52:51.707823 1064 tablet_bootstrap.cc:654] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.709132 1257 raft_consensus.cc:515] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.713184 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:51.715603 792 leader_election.cc:343] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:51.718567 382 tablet_service.cc:1558] Processing DeleteTablet for tablet f353e1e07485400a964f8e05b0691cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:51.719950 384 tablet_service.cc:1558] Processing DeleteTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:48 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:51.721356 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:51.722054 676 raft_consensus.cc:3060] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:51.722985 1257 leader_election.cc:290] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:51.723320 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 26) took 1431 ms (client timeout 1000 ms). Trace:
I20260810 07:52:51.724830 635 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.726565 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5eed268ee545ee91420cd792608b23 with delete_type TABLET_DATA_DELETED (replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.726774 868 tablet_service.cc:1558] Processing DeleteTablet for tablet f353e1e07485400a964f8e05b0691cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:51.727830 32470 rpcz_store.cc:269] 0810 07:52:50.272781 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.392318 (+119537us) service_pool.cc:224] Handling call
0810 07:52:51.704188 (+1311870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:50.963176 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:50.963653 (+ 477us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:50.963683 (+ 30us) write_op.cc:435] Acquired schema lock
0810 07:52:50.963699 (+ 16us) tablet.cc:662] Decoding operations
0810 07:52:50.964013 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:50.964055 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:50.964083 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:50.964250 (+ 167us) tablet.cc:701] Row locks acquired
0810 07:52:50.964264 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:50.964392 (+ 128us) write_op.cc:270] Start()
0810 07:52:50.964484 (+ 92us) write_op.cc:276] Timestamp: P: 1786348370964349 usec, L: 0
0810 07:52:50.964503 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:52:50.965149 (+ 646us) log.cc:844] Serialized 533 byte log entry
0810 07:52:51.096175 (+131026us) op_driver.cc:464] REPLICATION: finished
0810 07:52:51.096451 (+ 276us) write_op.cc:301] APPLY: starting
0810 07:52:51.096524 (+ 73us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:51.096636 (+ 112us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:51.096893 (+ 257us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:51.097164 (+ 271us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:51.097351 (+ 187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:51.097413 (+ 62us) write_op.cc:312] APPLY: finished
0810 07:52:51.097625 (+ 212us) log.cc:844] Serialized 23 byte log entry
0810 07:52:51.097811 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:51.097909 (+ 98us) write_op.cc:454] Released schema lock
0810 07:52:51.097946 (+ 37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":104,"apply.run_cpu_time_us":1796,"apply.run_wall_time_us":8618,"num_ops":1,"prepare.queue_time_us":46915,"prepare.run_cpu_time_us":2319,"prepare.run_wall_time_us":126155,"replication_time_us":131553}]]}
I20260810 07:52:51.721879 638 tablet_service.cc:1558] Processing DeleteTablet for tablet f353e1e07485400a964f8e05b0691cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.743669 1064 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:51.744185 1064 ts_tablet_manager.cc:1403] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.128s user 0.010s sys 0.005s
I20260810 07:52:51.748368 1064 raft_consensus.cc:359] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.749073 1064 raft_consensus.cc:385] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.748641 638 tablet_service.cc:1558] Processing DeleteTablet for tablet d6f37bba532b4d8bbc00e6d91f4c1820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.747673 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3ef130273507459b99d40cdeae4ac31f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:51.749317 1064 raft_consensus.cc:740] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.750029 1064 consensus_queue.cc:260] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.750478 645 tablet_service.cc:1558] Processing DeleteTablet for tablet a56f09a245fe4c1ea4e391166eaf3d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:48 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:51.750676 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ef130273507459b99d40cdeae4ac31f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:51.751781 676 raft_consensus.cc:2468] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:51.755247 792 leader_election.cc:304] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:51.756491 1257 raft_consensus.cc:2804] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:51.757040 1257 raft_consensus.cc:697] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:51.757812 1257 consensus_queue.cc:237] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.762126 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b5eed268ee545ee91420cd792608b23 with delete_type TABLET_DATA_DELETED (replaced by tablet e69ea1aae7784e75b7a7b3f5d67c1658) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.764989 868 tablet_service.cc:1511] Processing CreateTablet for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (DEFAULT_TABLE table=default.Table_680195385fd2411baf85a6a49bab6feb [id=ee979df2f5bc4651a69dfee31d80590f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.766103 864 tablet_service.cc:1558] Processing DeleteTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.769145 1064 ts_tablet_manager.cc:1434] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.025s user 0.005s sys 0.004s
I20260810 07:52:51.768814 384 tablet_service.cc:1511] Processing CreateTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 (DEFAULT_TABLE table=default.Table_29c70477feb94d599ff0c03f584bc04b [id=5fe5c44770974b8eb4f3b55096d25eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.771046 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09712cfebab34e56ba844e7aa0e8d152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.771557 1265 ts_tablet_manager.cc:1929] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:51.771890 1265 log.cc:1199] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/3f679d04625b4c0690a4ce8ea2a5e87b
I20260810 07:52:51.774399 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.775683 1437 tablet_replica.cc:333] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:51.776333 645 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.777099 1263 ts_tablet_manager.cc:1929] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:51.777606 1263 log.cc:1199] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:51.778889 1263 ts_tablet_manager.cc:1950] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:51.778769 638 tablet_service.cc:1511] Processing CreateTablet for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (DEFAULT_TABLE table=default.Table_680195385fd2411baf85a6a49bab6feb [id=ee979df2f5bc4651a69dfee31d80590f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.781332 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.781514 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 61) took 1168 ms. Trace:
I20260810 07:52:51.781881 1263 rpcz_store.cc:276] 0810 07:52:50.613221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.613346 (+ 125us) service_pool.cc:224] Handling call
0810 07:52:51.408970 (+795624us) ts_tablet_manager.cc:1151] Deleting tablet e9a1bcc922db45ad9f5f0edac6c7b5d5
0810 07:52:51.409029 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:51.598451 (+189422us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.648655 (+ 50204us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:51.781448 (+132793us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":794898}
I20260810 07:52:51.731563 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_746adafe126742f79e73baaf7b1fcfa7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:51.782697 1047 tablet_bootstrap.cc:654] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:51.785842 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_746adafe126742f79e73baaf7b1fcfa7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:51.788427 1064 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:51.796583 1331 raft_consensus.cc:493] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.797122 1071 tablet_bootstrap.cc:654] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.797087 1331 raft_consensus.cc:515] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:51.799245 1331 leader_election.cc:290] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.800601 1437 raft_consensus.cc:2243] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:51.801364 1437 raft_consensus.cc:2272] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:51.802352 309 leader_election.cc:343] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:51.700531 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a54449310a7f4844938ec08cfbf054d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:51.831035 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a54449310a7f4844938ec08cfbf054d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:51.784690 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]) successfully deleted
W20260810 07:52:51.833447 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 48779.6us
I20260810 07:52:51.835376 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.838024 380 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:51.842392 382 tablet_service.cc:1558] Processing DeleteTablet for tablet e44b480f5f3d4f239eae2586c1d2001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:51.843757 1431 tablet.cc:1719] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:51.845422 1408 tablet.cc:1719] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Skipping requested alter to schema version 2, tablet already version 2
W20260810 07:52:51.853165 1440 log.cc:927] Time spent T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260810 07:52:51.852898 864 tablet_service.cc:1511] Processing CreateTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 (DEFAULT_TABLE table=default.Table_29c70477feb94d599ff0c03f584bc04b [id=5fe5c44770974b8eb4f3b55096d25eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.854671 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09712cfebab34e56ba844e7aa0e8d152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.854147 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:51.795506 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 16) took 1822 ms (client timeout 1000 ms). Trace:
W20260810 07:52:51.856252 32468 rpcz_store.cc:269] 0810 07:52:49.972650 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.972818 (+ 168us) service_pool.cc:224] Handling call
0810 07:52:51.795427 (+1822609us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":10}
I20260810 07:52:51.856998 1064 tablet_bootstrap.cc:654] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:51.869565 1430 tablet.cc:1719] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:51.870549 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.877983 872 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:51.884291 872 tablet_service.cc:1511] Processing CreateTablet for tablet 7c075fdf1055477db9f2d56324f12a8a (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.885798 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c075fdf1055477db9f2d56324f12a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.890283 1431 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.891301 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b4772c3359a426e9f105760afff40cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:45 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.892616 382 tablet_service.cc:1511] Processing CreateTablet for tablet 7c075fdf1055477db9f2d56324f12a8a (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.893901 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c075fdf1055477db9f2d56324f12a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.889976 1064 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:51.898733 1064 ts_tablet_manager.cc:1403] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.111s user 0.011s sys 0.005s
I20260810 07:52:51.898672 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d397310bcfd2470e95fc4b0654749aa4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:51.816039 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 17) took 2528 ms (client timeout 1000 ms). Trace:
I20260810 07:52:51.819562 32469 master_service.cc:960] ReplaceTablet: received request to replace tablet 5b76191c5bed41509344376f1810e7aa from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:51.691409 32463 rpcz_store.cc:276] 0810 07:52:49.180919 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.209894 (+ 28975us) service_pool.cc:224] Handling call
0810 07:52:49.210983 (+ 1089us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:49.211755 (+ 772us) catalog_manager.cc:3751] Apply alter schema
0810 07:52:49.212887 (+ 1132us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:52:49.213073 (+ 186us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:52:49.584392 (+371319us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:52:51.690806 (+2106414us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:49.245244 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:49.246954 (+ 1710us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:49.246979 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:52:49.246992 (+ 13us) tablet.cc:662] Decoding operations
0810 07:52:49.247214 (+ 222us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:49.247245 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:49.247266 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:49.247415 (+ 149us) tablet.cc:701] Row locks acquired
0810 07:52:49.247429 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:49.247533 (+ 104us) write_op.cc:270] Start()
0810 07:52:49.247615 (+ 82us) write_op.cc:276] Timestamp: P: 1786348369247462 usec, L: 0
0810 07:52:49.247630 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:49.248149 (+ 519us) log.cc:844] Serialized 404 byte log entry
0810 07:52:49.545080 (+296931us) op_driver.cc:464] REPLICATION: finished
0810 07:52:49.574823 (+ 29743us) write_op.cc:301] APPLY: starting
0810 07:52:49.574908 (+ 85us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:49.575018 (+ 110us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:49.575213 (+ 195us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:49.575419 (+ 206us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:49.575650 (+ 231us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:49.575733 (+ 83us) write_op.cc:312] APPLY: finished
0810 07:52:49.576028 (+ 295us) log.cc:844] Serialized 22 byte log entry
0810 07:52:49.576288 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:49.576423 (+ 135us) write_op.cc:454] Released schema lock
0810 07:52:49.576466 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:49.589331 (+ 12865us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:49.589370 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":29410,"apply.run_cpu_time_us":2026,"apply.run_wall_time_us":14469,"mutex_wait_us":11835,"num_ops":1,"prepare.queue_time_us":30927,"prepare.run_cpu_time_us":2073,"prepare.run_wall_time_us":82218,"raft.queue_time_us":38442,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":16,"replication_time_us":297330}]]}
I20260810 07:52:51.908821 1064 raft_consensus.cc:359] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.909638 1064 raft_consensus.cc:385] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.909967 1064 raft_consensus.cc:740] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.772624 1265 ts_tablet_manager.cc:1950] T 3f679d04625b4c0690a4ce8ea2a5e87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:51.910766 1064 consensus_queue.cc:260] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.916886 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]) successfully deleted
I20260810 07:52:51.919564 1064 ts_tablet_manager.cc:1434] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20260810 07:52:51.921859 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet acf740de7abb40bfabfc3a7859cc159b from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:51.929214 1265 tablet_replica.cc:333] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:51.930605 1071 tablet_bootstrap.cc:492] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:51.931010 1047 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:51.931437 1047 ts_tablet_manager.cc:1403] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.344s user 0.009s sys 0.009s
I20260810 07:52:51.917047 1430 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:51.933878 646 tablet_service.cc:1511] Processing CreateTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 (DEFAULT_TABLE table=default.Table_29c70477feb94d599ff0c03f584bc04b [id=5fe5c44770974b8eb4f3b55096d25eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.935174 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09712cfebab34e56ba844e7aa0e8d152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.924090 645 tablet_service.cc:1558] Processing DeleteTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c075fdf1055477db9f2d56324f12a8a) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:51.928490 1390 consensus_queue.cc:1048] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:51.953474 868 tablet_service.cc:1511] Processing CreateTablet for tablet 1f8941c49c0a4fba9f7de2aabb790901 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.967000 1047 raft_consensus.cc:359] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.967736 1047 raft_consensus.cc:385] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:51.968015 1047 raft_consensus.cc:740] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:51.968623 1257 raft_consensus.cc:493] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:51.968647 1047 consensus_queue.cc:260] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.969103 1257 raft_consensus.cc:515] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:51.971305 1257 leader_election.cc:290] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:51.973701 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:51.879034 32461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 18) took 1583 ms (client timeout 1000 ms). Trace:
W20260810 07:52:51.974817 32461 rpcz_store.cc:269] 0810 07:52:50.296028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.446198 (+150170us) service_pool.cc:224] Handling call
0810 07:52:50.784531 (+338333us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:51.878922 (+1094391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16237,"reported_tablets":13,"spinlock_wait_cycles":528672640}
I20260810 07:52:51.975522 676 raft_consensus.cc:2468] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:52:51.877394 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 17) took 835 ms (client timeout 1000 ms). Trace:
W20260810 07:52:51.976902 792 leader_election.cc:343] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:51.976863 32465 rpcz_store.cc:269] 0810 07:52:51.042570 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.071982 (+ 29412us) service_pool.cc:224] Handling call
0810 07:52:51.084685 (+ 12703us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:51.877323 (+792638us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6935,"reported_tablets":14,"spinlock_wait_cycles":10033024}
I20260810 07:52:51.989106 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f8941c49c0a4fba9f7de2aabb790901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.993285 1047 ts_tablet_manager.cc:1434] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.062s user 0.006s sys 0.002s
I20260810 07:52:51.994261 639 tablet_service.cc:1511] Processing CreateTablet for tablet 7c075fdf1055477db9f2d56324f12a8a (DEFAULT_TABLE table=default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:51.996924 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c075fdf1055477db9f2d56324f12a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.000115 1064 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:52.003818 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_247ca9be54cb47c0b9d2f8aee9ac9f3b" } modify_external_catalogs: true
I20260810 07:52:52.005379 1265 raft_consensus.cc:2243] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.006456 1265 raft_consensus.cc:2272] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.006613 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d0c496eafbd142de89df899f2a939e6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:52.008798 1265 ts_tablet_manager.cc:1916] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:52.009778 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0c496eafbd142de89df899f2a939e6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:52.013717 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_395605094533475e8e04175ec53c96bb" } new_table_name: "default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98" modify_external_catalogs: true
I20260810 07:52:52.024760 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]). Expected version 2 got 0
I20260810 07:52:52.025277 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 5b76191c5bed41509344376f1810e7aa
I20260810 07:52:51.951604 380 tablet_service.cc:1511] Processing CreateTablet for tablet 1f8941c49c0a4fba9f7de2aabb790901 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.026539 1071 ts_tablet_manager.cc:1403] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.350s user 0.005s sys 0.008s
I20260810 07:52:52.027210 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b76191c5bed41509344376f1810e7aa with delete_type TABLET_DATA_DELETED (replaced by tablet 4d0b9b2012234cb19d337423b0f316af) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:52.027560 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f8941c49c0a4fba9f7de2aabb790901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:51.994321 1047 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:52.029192 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b76191c5bed41509344376f1810e7aa with delete_type TABLET_DATA_DELETED (replaced by tablet 4d0b9b2012234cb19d337423b0f316af) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:52.029726 32469 catalog_manager.cc:6807] ReplaceTablet: tablet 5b76191c5bed41509344376f1810e7aa deleted and replaced by tablet 4d0b9b2012234cb19d337423b0f316af
I20260810 07:52:51.975766 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.031235 1071 raft_consensus.cc:359] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.031900 1071 raft_consensus.cc:385] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.032151 1071 raft_consensus.cc:740] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.032838 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_247ca9be54cb47c0b9d2f8aee9ac9f3b" } modify_external_catalogs: true
I20260810 07:52:52.011602 377 tablet_service.cc:1511] Processing CreateTablet for tablet c292cb975c504ad8abf03e913140c7a9 (DEFAULT_TABLE table=default.Table_128834a6bbbe4f6da862be15162b8562 [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.035104 32460 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_480a23c1ffaa4896a093c62ceff5822e" } modify_external_catalogs: true
I20260810 07:52:52.035900 32460 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_480a23c1ffaa4896a093c62ceff5822e" } modify_external_catalogs: true
I20260810 07:52:51.978437 1437 ts_tablet_manager.cc:1916] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:51.900740 32464 rpcz_store.cc:269] 0810 07:52:49.287334 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:49.459596 (+172262us) service_pool.cc:224] Handling call
0810 07:52:49.546303 (+ 86707us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:51.815775 (+2269472us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:50.799023 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:50.799468 (+ 445us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:50.799490 (+ 22us) write_op.cc:435] Acquired schema lock
0810 07:52:50.799502 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:50.799867 (+ 365us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:50.799901 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:50.799922 (+ 21us) tablet.cc:685] Acquiring locks for 3 operations
0810 07:52:50.800734 (+ 812us) tablet.cc:701] Row locks acquired
0810 07:52:50.800751 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:52:50.800856 (+ 105us) write_op.cc:270] Start()
0810 07:52:50.800934 (+ 78us) write_op.cc:276] Timestamp: P: 1786348370800787 usec, L: 0
0810 07:52:50.800950 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:52:50.801571 (+ 621us) log.cc:844] Serialized 1292 byte log entry
0810 07:52:50.893517 (+ 91946us) op_driver.cc:464] REPLICATION: finished
0810 07:52:50.900923 (+ 7406us) write_op.cc:301] APPLY: starting
0810 07:52:50.900983 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:50.901114 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:50.901209 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:50.901495 (+ 286us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:50.901631 (+ 136us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0810 07:52:50.901677 (+ 46us) write_op.cc:312] APPLY: finished
0810 07:52:50.901887 (+ 210us) log.cc:844] Serialized 34 byte log entry
0810 07:52:50.902041 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:50.902131 (+ 90us) write_op.cc:454] Released schema lock
0810 07:52:50.902163 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":75925,"reported_tablets":10,"spinlock_wait_cycles":149376896,"child_traces":[["op",{"apply.queue_time_us":7215,"apply.run_cpu_time_us":1473,"apply.run_wall_time_us":1473,"num_ops":3,"prepare.queue_time_us":48303,"prepare.run_cpu_time_us":2055,"prepare.run_wall_time_us":4185,"raft.queue_time_us":641,"raft.run_cpu_time_us":22,"raft.run_wall_time_us":23,"replication_time_us":92453}]]}
I20260810 07:52:52.056838 1064 tablet_bootstrap.cc:654] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.061036 1154 raft_consensus.cc:493] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.062079 1154 raft_consensus.cc:515] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.084438 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c292cb975c504ad8abf03e913140c7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.098376 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_5b06ee13a0114e059c520a2edd38edf4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:52.100126 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b06ee13a0114e059c520a2edd38edf4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:52.113231 1154 leader_election.cc:290] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.114351 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 63) took 1431 ms. Trace:
I20260810 07:52:52.114737 1263 rpcz_store.cc:276] 0810 07:52:50.683080 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:50.685364 (+ 2284us) service_pool.cc:224] Handling call
0810 07:52:52.113914 (+1428550us) ts_tablet_manager.cc:1151] Deleting tablet 4ec00a494816409684205c7f4435a079
0810 07:52:52.113990 (+ 76us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.114289 (+ 299us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1402032}
I20260810 07:52:52.115815 1263 tablet_replica.cc:333] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.124691 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.124691 308 leader_election.cc:304] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:52.126533 1331 raft_consensus.cc:2804] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:52.126894 1331 raft_consensus.cc:493] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:52.127194 1331 raft_consensus.cc:3060] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.129568 1263 raft_consensus.cc:2243] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:52.130050 1263 raft_consensus.cc:2272] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.138633 1408 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260810 07:52:52.058440 792 leader_election.cc:343] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.135564 869 tablet_service.cc:1511] Processing CreateTablet for tablet c292cb975c504ad8abf03e913140c7a9 (DEFAULT_TABLE table=default.Table_128834a6bbbe4f6da862be15162b8562 [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.140996 792 leader_election.cc:304] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:52.141592 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c292cb975c504ad8abf03e913140c7a9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:52.142192 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 83800.6us
I20260810 07:52:52.143790 1257 raft_consensus.cc:2749] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:52.139463 871 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfffdd01524eada0a85c1a525325a0 (DEFAULT_TABLE table=default.Table_97132925a8444a3d9c702a3ae7cbb209 [id=833396f0e1d74f589d03c1c918dd843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.144598 1257 raft_consensus.cc:493] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.145486 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfffdd01524eada0a85c1a525325a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.145320 1257 raft_consensus.cc:515] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.148324 1257 leader_election.cc:290] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:52.151918 1257 raft_consensus.cc:493] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.152586 1257 raft_consensus.cc:515] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.157270 1257 leader_election.cc:290] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:52.161170 1331 raft_consensus.cc:515] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:52.163256 384 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfffdd01524eada0a85c1a525325a0 (DEFAULT_TABLE table=default.Table_97132925a8444a3d9c702a3ae7cbb209 [id=833396f0e1d74f589d03c1c918dd843a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:52.167546 309 leader_election.cc:343] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:52.172199 1263 ts_tablet_manager.cc:1916] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.172371 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:52.173408 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d3fcf79a104c47a6fd10fc34f7ce08" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:52.118794 32464 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_536541f9abf14594aaf6e26a1163d266" } new_table_name: "default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a" modify_external_catalogs: true
W20260810 07:52:52.121723 32467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 19) took 769 ms (client timeout 1000 ms). Trace:
W20260810 07:52:52.181057 32467 rpcz_store.cc:269] 0810 07:52:51.352519 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.483868 (+131349us) service_pool.cc:224] Handling call
0810 07:52:51.736289 (+252421us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:51.736317 (+ 28us) write_op.cc:454] Released schema lock
0810 07:52:52.121659 (+385342us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:51.702769 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:51.703258 (+ 489us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:51.703283 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:52:51.703295 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:51.703775 (+ 480us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:51.703816 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:51.703839 (+ 23us) tablet.cc:685] Acquiring locks for 4 operations
0810 07:52:51.704172 (+ 333us) tablet.cc:701] Row locks acquired
0810 07:52:51.704186 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:51.704307 (+ 121us) write_op.cc:270] Start()
0810 07:52:51.704394 (+ 87us) write_op.cc:276] Timestamp: P: 1786348371704251 usec, L: 0
0810 07:52:51.704410 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:52:51.704990 (+ 580us) log.cc:844] Serialized 1674 byte log entry
0810 07:52:51.717809 (+ 12819us) op_driver.cc:464] REPLICATION: finished
0810 07:52:51.722919 (+ 5110us) write_op.cc:301] APPLY: starting
0810 07:52:51.723001 (+ 82us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:51.723229 (+ 228us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:51.723539 (+ 310us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:51.724171 (+ 632us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:51.724421 (+ 250us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0810 07:52:51.724489 (+ 68us) write_op.cc:312] APPLY: finished
0810 07:52:51.724735 (+ 246us) log.cc:844] Serialized 41 byte log entry
0810 07:52:51.724956 (+ 221us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:51.725083 (+ 127us) write_op.cc:454] Released schema lock
0810 07:52:51.725141 (+ 58us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":4820,"apply.run_cpu_time_us":2570,"apply.run_wall_time_us":2569,"num_ops":4,"prepare.queue_time_us":100,"prepare.run_cpu_time_us":2576,"prepare.run_wall_time_us":2575,"raft.queue_time_us":4039,"raft.run_cpu_time_us":664,"raft.run_wall_time_us":665,"replication_time_us":13202}]]}
I20260810 07:52:52.185140 646 tablet_service.cc:1511] Processing CreateTablet for tablet 1f8941c49c0a4fba9f7de2aabb790901 (DEFAULT_TABLE table=default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.186813 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f8941c49c0a4fba9f7de2aabb790901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.186616 1071 consensus_queue.cc:260] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.189029 644 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:52.189320 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b76191c5bed41509344376f1810e7aa with delete_type TABLET_DATA_DELETED (replaced by tablet 4d0b9b2012234cb19d337423b0f316af) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:52.195475 1047 tablet_bootstrap.cc:654] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.195621 1331 leader_election.cc:290] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:52.180471 573 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:52.201489 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 4ec00a494816409684205c7f4435a079 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ec00a494816409684205c7f4435a079 already in progress: creating tablet
I20260810 07:52:52.207103 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.208478 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9d3fcf79a104c47a6fd10fc34f7ce08" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.221475 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfffdd01524eada0a85c1a525325a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.237344 32465 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 113730acd72141f4b9fa4543d94d930b
I20260810 07:52:52.239853 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0f57e2f93400402d8c7458cf70c65ccc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:52.244230 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:52.246002 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:52.250619 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f57e2f93400402d8c7458cf70c65ccc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:52.250795 1454 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:52:52.261127 573 consensus_peers.cc:597] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:52.251295 1454 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.264027 1390 consensus_queue.cc:1048] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260810 07:52:52.271626 1454 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.260386 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d397310bcfd2470e95fc4b0654749aa4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:52.273480 676 raft_consensus.cc:3060] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.271533 635 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfffdd01524eada0a85c1a525325a0 (DEFAULT_TABLE table=default.Table_97132925a8444a3d9c702a3ae7cbb209 [id=833396f0e1d74f589d03c1c918dd843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.288419 645 tablet_service.cc:1511] Processing CreateTablet for tablet c292cb975c504ad8abf03e913140c7a9 (DEFAULT_TABLE table=default.Table_128834a6bbbe4f6da862be15162b8562 [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:52.290391 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c292cb975c504ad8abf03e913140c7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.294499 1071 ts_tablet_manager.cc:1434] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.268s user 0.003s sys 0.006s
I20260810 07:52:52.301290 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfffdd01524eada0a85c1a525325a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:52.320032 1071 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:52.321550 1437 ts_tablet_manager.cc:1929] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:52.322105 1437 log.cc:1199] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/f353e1e07485400a964f8e05b0691cc6
I20260810 07:52:52.323858 1437 ts_tablet_manager.cc:1950] T f353e1e07485400a964f8e05b0691cc6 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:52.325996 1437 tablet_replica.cc:333] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:52.327052 1437 raft_consensus.cc:2243] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:52.327787 1437 raft_consensus.cc:2272] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.330533 1437 ts_tablet_manager.cc:1916] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.348649 1263 ts_tablet_manager.cc:1929] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:52.349126 1263 log.cc:1199] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:52.355521 1047 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:52.355962 1047 ts_tablet_manager.cc:1403] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.362s user 0.004s sys 0.022s
I20260810 07:52:52.362052 1437 ts_tablet_manager.cc:1929] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:52.366102 1437 log.cc:1199] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/a56f09a245fe4c1ea4e391166eaf3d3d
I20260810 07:52:52.349921 1263 ts_tablet_manager.cc:1950] T c3675c5bae7a42769223ef5b2fc433b3 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:52.300423 676 raft_consensus.cc:2468] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:52:52.370760 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 108822us
I20260810 07:52:52.374191 308 leader_election.cc:304] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:52.376025 1455 raft_consensus.cc:2804] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:52.372664 1437 ts_tablet_manager.cc:1950] T a56f09a245fe4c1ea4e391166eaf3d3d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:52.376936 1455 raft_consensus.cc:697] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:52:52.377981 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.379123 1455 consensus_queue.cc:237] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:52:52.380126 1047 raft_consensus.cc:359] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.381083 1047 raft_consensus.cc:385] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.382576 1445 tablet.cc:1724] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:52.392355 1071 tablet_bootstrap.cc:654] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.396595 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 64) took 1261 ms. Trace:
I20260810 07:52:52.397190 1263 rpcz_store.cc:276] 0810 07:52:51.134937 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.232435 (+ 97498us) service_pool.cc:224] Handling call
0810 07:52:52.115666 (+883231us) ts_tablet_manager.cc:1151] Deleting tablet c3675c5bae7a42769223ef5b2fc433b3
0810 07:52:52.115707 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.303988 (+188281us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.348508 (+ 44520us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.396492 (+ 47984us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12757,"tablet-delete.queue_time_us":882354}
W20260810 07:52:52.398897 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 105982us
I20260810 07:52:52.401491 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]) successfully deleted
I20260810 07:52:52.403863 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet a56f09a245fe4c1ea4e391166eaf3d3d (table default.Table_ae672329134f4bbf8959c4cc2a85982c [id=e473bca3a2c845319c20da3403ffcea4]) successfully deleted
I20260810 07:52:52.422186 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 1ba026e367ea487fb0dc8aef5d5db606
I20260810 07:52:52.395081 1437 tablet_replica.cc:333] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:52.420245 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet c3675c5bae7a42769223ef5b2fc433b3 (table default.Table_3a887c70a4ef45ccace031aa6fa907de [id=43a67562d28b442a86908e5e13f18832]) successfully deleted
I20260810 07:52:52.424527 1263 tablet_replica.cc:333] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.429289 32460 master_service.cc:960] ReplaceTablet: received request to replace tablet 92e2678ab165486dbbb69c7c183a7fa4 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:52.431553 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:52.437418 1408 tablet.cc:1724] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:52.439684 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:52.445493 573 leader_election.cc:343] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.381455 1047 raft_consensus.cc:740] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.453920 1047 consensus_queue.cc:260] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.460140 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet acf740de7abb40bfabfc3a7859cc159b
I20260810 07:52:52.462409 32468 catalog_manager.cc:6807] ReplaceTablet: tablet acf740de7abb40bfabfc3a7859cc159b deleted and replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed
I20260810 07:52:52.444624 1263 raft_consensus.cc:2243] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.462957 32468 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 111) took 1139 ms. Trace:
I20260810 07:52:52.463150 1047 ts_tablet_manager.cc:1434] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.107s user 0.009s sys 0.000s
I20260810 07:52:52.463305 1263 raft_consensus.cc:2272] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.463146 384 tablet_service.cc:1558] Processing DeleteTablet for tablet acf740de7abb40bfabfc3a7859cc159b with delete_type TABLET_DATA_DELETED (replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:52.465467 573 consensus_peers.cc:597] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:52.463315 32468 rpcz_store.cc:276] 0810 07:52:51.323674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.921564 (+597890us) service_pool.cc:224] Handling call
0810 07:52:52.011716 (+ 90152us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ed41 00007f30cf440d37 00007f30cf440cc2 00007f30cf440bf1 00007f30cf440b6c 00007f30cf43fe22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:52.084845 (+ 73129us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.084887 (+ 42us) write_op.cc:454] Released schema lock
0810 07:52:52.462829 (+377942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.037376 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.037850 (+ 474us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.037878 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:52:52.037891 (+ 13us) tablet.cc:662] Decoding operations
0810 07:52:52.038259 (+ 368us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.038310 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.038342 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:52.038576 (+ 234us) tablet.cc:701] Row locks acquired
0810 07:52:52.038599 (+ 23us) write_op.cc:260] PREPARE: finished
0810 07:52:52.038741 (+ 142us) write_op.cc:270] Start()
0810 07:52:52.038838 (+ 97us) write_op.cc:276] Timestamp: P: 1786348372038681 usec, L: 0
0810 07:52:52.038852 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.039502 (+ 650us) log.cc:844] Serialized 628 byte log entry
0810 07:52:52.052748 (+ 13246us) op_driver.cc:464] REPLICATION: finished
0810 07:52:52.054410 (+ 1662us) write_op.cc:301] APPLY: starting
0810 07:52:52.054496 (+ 86us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:52.054706 (+ 210us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:52.054931 (+ 225us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:52.055535 (+ 604us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:52.055966 (+ 431us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:52.056042 (+ 76us) write_op.cc:312] APPLY: finished
0810 07:52:52.056362 (+ 320us) log.cc:844] Serialized 29 byte log entry
0810 07:52:52.073637 (+ 17275us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.073755 (+ 118us) write_op.cc:454] Released schema lock
0810 07:52:52.073975 (+ 220us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":63547648,"child_traces":[["op",{"apply.queue_time_us":1401,"apply.run_cpu_time_us":2935,"apply.run_wall_time_us":19907,"num_ops":2,"prepare.queue_time_us":303,"prepare.run_cpu_time_us":2473,"prepare.run_wall_time_us":4592,"raft.queue_time_us":891,"raft.run_cpu_time_us":601,"raft.run_wall_time_us":602,"replication_time_us":13719}]]}
I20260810 07:52:52.474298 1263 ts_tablet_manager.cc:1916] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:52.477106 573 consensus_peers.cc:597] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:52.478178 639 tablet_service.cc:1558] Processing DeleteTablet for tablet acf740de7abb40bfabfc3a7859cc159b with delete_type TABLET_DATA_DELETED (replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:52.482795 32470 catalog_manager.cc:5719] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:52.485143 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]). Expected version 2 got 0
I20260810 07:52:52.486318 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 92e2678ab165486dbbb69c7c183a7fa4 (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]). Expected version 1 got 0
I20260810 07:52:52.168713 1265 ts_tablet_manager.cc:1929] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:52.490139 1265 log.cc:1199] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/f353e1e07485400a964f8e05b0691cc6
I20260810 07:52:52.490904 1265 ts_tablet_manager.cc:1950] T f353e1e07485400a964f8e05b0691cc6 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:52.498055 1257 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.498550 1257 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.500512 1257 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:52.506582 1265 tablet_replica.cc:333] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:52.507514 1265 raft_consensus.cc:2243] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:52.507985 1265 raft_consensus.cc:2272] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.455061 1064 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:52.509918 1064 ts_tablet_manager.cc:1403] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.510s user 0.006s sys 0.009s
W20260810 07:52:52.175303 792 leader_election.cc:343] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.472612 864 tablet_service.cc:1558] Processing DeleteTablet for tablet acf740de7abb40bfabfc3a7859cc159b with delete_type TABLET_DATA_DELETED (replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:52.525095 1064 raft_consensus.cc:359] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.525931 1064 raft_consensus.cc:385] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.526260 1064 raft_consensus.cc:740] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.527073 1064 consensus_queue.cc:260] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.529325 1064 ts_tablet_manager.cc:1434] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
W20260810 07:52:52.516562 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 341270us
W20260810 07:52:52.533152 792 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.534982 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.535266 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:52.535925 676 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:52.536269 1331 raft_consensus.cc:493] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.536753 1331 raft_consensus.cc:515] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.540251 1331 leader_election.cc:290] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:52.541766 308 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: a1373ebbd3ba4262806bfc17823d25b5
I20260810 07:52:52.546634 1263 ts_tablet_manager.cc:1929] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:52.453312 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:52.547049 1263 log.cc:1199] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/3f679d04625b4c0690a4ce8ea2a5e87b
I20260810 07:52:52.548501 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:52.549142 897 raft_consensus.cc:2468] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:52.549358 1047 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:52:52.551690 309 leader_election.cc:343] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: a1373ebbd3ba4262806bfc17823d25b5
I20260810 07:52:52.552130 309 leader_election.cc:304] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:52.566138 1064 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:52.547832 1263 ts_tablet_manager.cc:1950] T 3f679d04625b4c0690a4ce8ea2a5e87b P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:52.576308 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 66) took 1341 ms. Trace:
I20260810 07:52:52.576697 1263 rpcz_store.cc:276] 0810 07:52:51.234802 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.235015 (+ 213us) service_pool.cc:224] Handling call
0810 07:52:52.423267 (+1188252us) ts_tablet_manager.cc:1151] Deleting tablet 3f679d04625b4c0690a4ce8ea2a5e87b
0810 07:52:52.423738 (+ 471us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.486704 (+ 62966us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.546498 (+ 59794us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.576211 (+ 29713us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1187263}
I20260810 07:52:52.578065 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 3f679d04625b4c0690a4ce8ea2a5e87b (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]) successfully deleted
I20260810 07:52:52.578325 1265 ts_tablet_manager.cc:1916] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.580551 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:52.581128 1047 tablet_bootstrap.cc:654] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.581499 1263 tablet_replica.cc:333] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.581348 891 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:52.582194 1263 raft_consensus.cc:2243] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.582980 1263 raft_consensus.cc:2272] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.588085 1480 raft_consensus.cc:2749] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:52.589569 1263 ts_tablet_manager.cc:1916] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:52.600318 792 leader_election.cc:343] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.600818 792 leader_election.cc:304] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:52.603415 1480 raft_consensus.cc:493] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.604178 1480 raft_consensus.cc:515] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.608023 1480 leader_election.cc:290] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:52.617079 308 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:52.624733 1263 ts_tablet_manager.cc:1929] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:52.625157 1263 log.cc:1199] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/e44b480f5f3d4f239eae2586c1d2001a
I20260810 07:52:52.625828 1263 ts_tablet_manager.cc:1950] T e44b480f5f3d4f239eae2586c1d2001a P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:52.628417 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet e44b480f5f3d4f239eae2586c1d2001a (table default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]) successfully deleted
I20260810 07:52:52.630288 1047 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:52.630760 1047 ts_tablet_manager.cc:1403] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.006s
I20260810 07:52:52.633612 1047 raft_consensus.cc:359] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.634455 1047 raft_consensus.cc:385] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.634697 1047 raft_consensus.cc:740] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.638865 32461 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 112) took 1241 ms. Trace:
I20260810 07:52:52.641010 1047 consensus_queue.cc:260] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.649945 1263 tablet_replica.cc:333] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.650686 1263 raft_consensus.cc:2243] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:52:52.651695 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 6b5eed268ee545ee91420cd792608b23 because the tablet was not found. No further retry: Not found: Tablet not found: 6b5eed268ee545ee91420cd792608b23
I20260810 07:52:52.652822 1437 raft_consensus.cc:2243] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.654693 1263 raft_consensus.cc:2272] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.655635 1437 pending_rounds.cc:70] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Trying to abort 1 pending ops.
I20260810 07:52:52.655951 1437 pending_rounds.cc:77] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882933282443264 op_type: NO_OP noop_request { }
I20260810 07:52:52.656674 1263 ts_tablet_manager.cc:1916] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:52.659967 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6226a2377864d47b14901df39677127 has no consensus state
I20260810 07:52:52.665885 32462 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 113) took 1002 ms. Trace:
I20260810 07:52:52.666507 1047 ts_tablet_manager.cc:1434] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.035s user 0.007s sys 0.001s
I20260810 07:52:52.667522 1047 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:52.670324 1071 tablet_bootstrap.cc:492] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:52.670840 1071 ts_tablet_manager.cc:1403] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.351s user 0.021s sys 0.049s
I20260810 07:52:52.656481 1437 raft_consensus.cc:2889] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:52.671825 1437 raft_consensus.cc:2272] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.639525 32461 rpcz_store.cc:276] 0810 07:52:51.396909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.976168 (+579259us) service_pool.cc:224] Handling call
0810 07:52:52.006487 (+ 30319us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c1c6 00007f30cf420034 00007f30cf41ffc2 00007f30cf41fef1 00007f30cf41fe6c 00007f30cf41f122 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:52.042557 (+ 36070us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:52:52.074764 (+ 32207us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:52:52.636459 (+561695us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:52:52.637280 (+ 821us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:52:52.638191 (+ 911us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.227291 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.241706 (+ 14415us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.241733 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:52:52.241745 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:52.242033 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.242074 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.242097 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:52.242315 (+ 218us) tablet.cc:701] Row locks acquired
0810 07:52:52.242329 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:52.242477 (+ 148us) write_op.cc:270] Start()
0810 07:52:52.242591 (+ 114us) write_op.cc:276] Timestamp: P: 1786348372242401 usec, L: 0
0810 07:52:52.242607 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.243234 (+ 627us) log.cc:844] Serialized 433 byte log entry
0810 07:52:52.631909 (+388675us) op_driver.cc:464] REPLICATION: finished
0810 07:52:52.632598 (+ 689us) write_op.cc:301] APPLY: starting
0810 07:52:52.632831 (+ 233us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:52.633071 (+ 240us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:52.633256 (+ 185us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:52.634229 (+ 973us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:52.634541 (+ 312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:52:52.634675 (+ 134us) write_op.cc:312] APPLY: finished
0810 07:52:52.635118 (+ 443us) log.cc:844] Serialized 29 byte log entry
0810 07:52:52.635422 (+ 304us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.635553 (+ 131us) write_op.cc:454] Released schema lock
0810 07:52:52.635624 (+ 71us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":64744320,"child_traces":[["op",{"apply.queue_time_us":252,"apply.run_cpu_time_us":3449,"apply.run_wall_time_us":8575,"num_ops":2,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":2238,"prepare.run_wall_time_us":16187,"replication_time_us":389032}]]}
I20260810 07:52:52.676695 1071 raft_consensus.cc:359] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.685456 1071 raft_consensus.cc:385] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.685822 1071 raft_consensus.cc:740] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.666505 32462 rpcz_store.cc:276] 0810 07:52:51.663237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.977075 (+313838us) service_pool.cc:224] Handling call
0810 07:52:52.013429 (+ 36354us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c846 00007f30cf429634 00007f30cf4295c2 00007f30cf4294f1 00007f30cf42946c 00007f30cf428722 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:52.068450 (+ 55021us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:52.071579 (+ 3129us) catalog_manager.cc:3751] Apply alter schema
0810 07:52:52.073219 (+ 1640us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:52:52.227482 (+154263us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:52:52.646010 (+418528us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:52:52.665514 (+ 19504us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.243585 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.243996 (+ 411us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.244018 (+ 22us) write_op.cc:435] Acquired schema lock
0810 07:52:52.244029 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:52.244267 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.244304 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.244324 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:52.244465 (+ 141us) tablet.cc:701] Row locks acquired
0810 07:52:52.244478 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:52.244605 (+ 127us) write_op.cc:270] Start()
0810 07:52:52.266666 (+ 22061us) write_op.cc:276] Timestamp: P: 1786348372244542 usec, L: 0
0810 07:52:52.266698 (+ 32us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.267541 (+ 843us) log.cc:844] Serialized 405 byte log entry
0810 07:52:52.641869 (+374328us) op_driver.cc:464] REPLICATION: finished
0810 07:52:52.643036 (+ 1167us) write_op.cc:301] APPLY: starting
0810 07:52:52.643187 (+ 151us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:52.643415 (+ 228us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:52.643654 (+ 239us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:52.644129 (+ 475us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:52.644342 (+ 213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:52.644411 (+ 69us) write_op.cc:312] APPLY: finished
0810 07:52:52.644870 (+ 459us) log.cc:844] Serialized 23 byte log entry
0810 07:52:52.645088 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.645166 (+ 78us) write_op.cc:454] Released schema lock
0810 07:52:52.645199 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":79634176,"child_traces":[["op",{"apply.queue_time_us":869,"apply.run_cpu_time_us":2705,"apply.run_wall_time_us":41331,"num_ops":1,"prepare.queue_time_us":14236,"prepare.run_cpu_time_us":2487,"prepare.run_wall_time_us":52102,"replication_time_us":374797}]]}
I20260810 07:52:52.686554 1071 consensus_queue.cc:260] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.678331 1064 tablet_bootstrap.cc:654] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.689898 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f4269fe23ea14f68b3bbf0770396ffc5" } modify_external_catalogs: true
I20260810 07:52:52.693450 1071 ts_tablet_manager.cc:1434] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.022s user 0.005s sys 0.004s
I20260810 07:52:52.694391 1071 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:52.693509 1437 ts_tablet_manager.cc:1916] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.700814 1459 raft_consensus.cc:493] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.701342 1459 raft_consensus.cc:515] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.708537 1263 ts_tablet_manager.cc:1929] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:52.712827 1263 log.cc:1199] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/f353e1e07485400a964f8e05b0691cc6
I20260810 07:52:52.713522 1064 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:52.714004 1064 ts_tablet_manager.cc:1403] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.155s user 0.014s sys 0.003s
I20260810 07:52:52.714839 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af5109e338844139444bb7111619195" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.714993 1459 leader_election.cc:290] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.713757 1263 ts_tablet_manager.cc:1950] T f353e1e07485400a964f8e05b0691cc6 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:52:52.716982 573 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.718545 1071 tablet_bootstrap.cc:654] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.721060 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 73) took 1005 ms. Trace:
I20260810 07:52:52.721442 1263 rpcz_store.cc:276] 0810 07:52:51.716093 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.721199 (+ 5106us) service_pool.cc:224] Handling call
0810 07:52:52.649749 (+928550us) ts_tablet_manager.cc:1151] Deleting tablet f353e1e07485400a964f8e05b0691cc6
0810 07:52:52.649795 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.669783 (+ 19988us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.708414 (+ 38631us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.720994 (+ 12580us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3388,"tablet-delete.queue_time_us":911526}
I20260810 07:52:52.722124 1257 raft_consensus.cc:2749] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:52.724936 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 124665us
I20260810 07:52:52.724849 1064 raft_consensus.cc:359] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.725492 1064 raft_consensus.cc:385] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.725766 1064 raft_consensus.cc:740] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.726539 1064 consensus_queue.cc:260] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.729315 1459 raft_consensus.cc:493] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.729835 1459 raft_consensus.cc:515] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:52.731086 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 179383us
I20260810 07:52:52.732641 1071 tablet_bootstrap.cc:492] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:52.733127 1071 ts_tablet_manager.cc:1403] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.007s
I20260810 07:52:52.733012 1263 tablet_replica.cc:333] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.733459 1459 leader_election.cc:290] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.733299 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:52.734411 1263 raft_consensus.cc:2243] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:52.734932 1263 raft_consensus.cc:2272] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:52.735623 573 leader_election.cc:343] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.739516 1064 ts_tablet_manager.cc:1434] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
I20260810 07:52:52.740828 1064 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:52.702611 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]) successfully deleted
I20260810 07:52:52.744458 1071 raft_consensus.cc:359] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.745334 1071 raft_consensus.cc:385] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.745625 1071 raft_consensus.cc:740] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.745760 1265 ts_tablet_manager.cc:1929] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:52.746184 1265 log.cc:1199] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/4af5109e338844139444bb7111619195
W20260810 07:52:52.746799 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 6b5eed268ee545ee91420cd792608b23 because the tablet was not found. No further retry: Not found: Tablet not found: 6b5eed268ee545ee91420cd792608b23
I20260810 07:52:52.746975 1265 ts_tablet_manager.cc:1950] T 4af5109e338844139444bb7111619195 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
W20260810 07:52:52.747743 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c3675c5bae7a42769223ef5b2fc433b3 because the tablet was not found. No further retry: Not found: Tablet not found: c3675c5bae7a42769223ef5b2fc433b3
W20260810 07:52:52.748404 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 4ec00a494816409684205c7f4435a079 because the tablet was not found. No further retry: Not found: Tablet not found: 4ec00a494816409684205c7f4435a079
I20260810 07:52:52.749660 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f4269fe23ea14f68b3bbf0770396ffc5" } modify_external_catalogs: true
I20260810 07:52:52.746260 1071 consensus_queue.cc:260] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.752123 1331 raft_consensus.cc:493] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.736896 1263 ts_tablet_manager.cc:1916] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.752832 1331 raft_consensus.cc:515] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.754729 1071 ts_tablet_manager.cc:1434] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260810 07:52:52.711016 32461 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_fce89a2685ea4f499401deb1152927b9" } modify_external_catalogs: true
I20260810 07:52:52.723033 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet f353e1e07485400a964f8e05b0691cc6 (table default.Table_0f6aa0ec81114d78abb2ad53a02bf930 [id=cb8f9731c5f14b198d4ef6fa1ca03cd1]) successfully deleted
I20260810 07:52:52.755470 32461 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_fce89a2685ea4f499401deb1152927b9" } modify_external_catalogs: true
W20260810 07:52:52.756343 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet d6f37bba532b4d8bbc00e6d91f4c1820 because the tablet was not found. No further retry: Not found: Tablet not found: d6f37bba532b4d8bbc00e6d91f4c1820
I20260810 07:52:52.757042 1331 leader_election.cc:290] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:52.757087 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet a56f09a245fe4c1ea4e391166eaf3d3d because the tablet was not found. No further retry: Not found: Tablet not found: a56f09a245fe4c1ea4e391166eaf3d3d
W20260810 07:52:52.725769 792 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.757726 792 leader_election.cc:304] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:52:52.758898 792 leader_election.cc:343] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.759044 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:52.760004 574 leader_election.cc:304] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:52.761067 1459 raft_consensus.cc:2804] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:52.761535 1459 raft_consensus.cc:493] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:52.762027 1459 raft_consensus.cc:3060] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.763760 1257 raft_consensus.cc:2749] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:52.765084 1071 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:52.770651 309 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:52.771267 309 leader_election.cc:304] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:52.773218 1196 consensus_queue.cc:1048] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:52.775153 1196 consensus_queue.cc:1048] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:52.777541 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba026e367ea487fb0dc8aef5d5db606" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:52.778427 1047 tablet_bootstrap.cc:654] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.779214 891 raft_consensus.cc:2393] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:52.780345 574 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:52.782343 1454 raft_consensus.cc:2804] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:52.782650 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 4af5109e338844139444bb7111619195 (table default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]) successfully deleted
I20260810 07:52:52.782724 1454 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:52.783064 1454 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:52.784013 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 because the tablet was not found. No further retry: Not found: Tablet not found: e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:52.787690 1459 raft_consensus.cc:515] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.779822 792 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:52.788816 1196 consensus_queue.cc:1048] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:52.790475 1064 tablet_bootstrap.cc:654] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.791334 1390 consensus_queue.cc:1048] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:52.792681 1257 raft_consensus.cc:2804] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:52.793113 1257 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:52.793495 1257 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.797526 1454 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.799819 1481 raft_consensus.cc:2749] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:52.782051 1265 tablet_replica.cc:333] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:52.807132 309 leader_election.cc:304] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:52.808945 1481 raft_consensus.cc:3060] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.813654 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:52.815912 573 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.805718 1459 leader_election.cc:290] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.813478 1265 raft_consensus.cc:2243] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:52.818981 1265 raft_consensus.cc:2272] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.821738 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:52.821552 1257 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:52.823283 573 leader_election.cc:343] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.824417 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:52.825240 1454 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.825258 894 raft_consensus.cc:2468] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:52.826750 1047 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:52.827219 1047 ts_tablet_manager.cc:1403] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.160s user 0.009s sys 0.003s
I20260810 07:52:52.830379 1265 ts_tablet_manager.cc:1916] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.830469 1047 raft_consensus.cc:359] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.831104 1047 raft_consensus.cc:385] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:52.831359 1047 raft_consensus.cc:740] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.839979 1481 raft_consensus.cc:2749] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:52.832309 1047 consensus_queue.cc:260] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.859408 1047 ts_tablet_manager.cc:1434] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
I20260810 07:52:52.860324 1047 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:52.863003 1257 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:52.864419 1263 ts_tablet_manager.cc:1929] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:52.865578 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "338e9b0e89d44295973d4003abd36649" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:52.866276 893 raft_consensus.cc:3060] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.866405 1263 log.cc:1199] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:52.867864 1263 ts_tablet_manager.cc:1950] T d397310bcfd2470e95fc4b0654749aa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:52.871778 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 78) took 1097 ms. Trace:
I20260810 07:52:52.876255 893 raft_consensus.cc:2468] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:52.878780 1263 rpcz_store.cc:276] 0810 07:52:51.774492 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.775941 (+ 1449us) service_pool.cc:224] Handling call
0810 07:52:52.732864 (+956923us) ts_tablet_manager.cc:1151] Deleting tablet d397310bcfd2470e95fc4b0654749aa4
0810 07:52:52.732901 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.840546 (+107645us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.864299 (+ 23753us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.871710 (+ 7411us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":955771}
I20260810 07:52:52.880538 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet d397310bcfd2470e95fc4b0654749aa4 (table default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]) successfully deleted
I20260810 07:52:52.881853 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 80) took 1030 ms. Trace:
I20260810 07:52:52.882256 1263 rpcz_store.cc:276] 0810 07:52:51.851716 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.853923 (+ 2207us) service_pool.cc:224] Handling call
0810 07:52:52.881523 (+1027600us) ts_tablet_manager.cc:1151] Deleting tablet 95fcae5b2e5448bf89b1c0587f46ad40
0810 07:52:52.881579 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.881796 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1026380}
W20260810 07:52:52.883698 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 95fcae5b2e5448bf89b1c0587f46ad40 because the tablet was not found. No further retry: Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:52.886384 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 81) took 1030 ms. Trace:
W20260810 07:52:52.888787 574 consensus_peers.cc:597] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:52.889334 1263 rpcz_store.cc:276] 0810 07:52:51.855772 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.888774 (+ 33002us) service_pool.cc:224] Handling call
0810 07:52:52.886087 (+997313us) ts_tablet_manager.cc:1151] Deleting tablet 0b4772c3359a426e9f105760afff40cb
0810 07:52:52.886139 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.886320 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":994052}
I20260810 07:52:52.890266 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:52.891254 893 raft_consensus.cc:2393] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
W20260810 07:52:52.891726 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0b4772c3359a426e9f105760afff40cb because the tablet was not found. No further retry: Not found: Tablet not found: 0b4772c3359a426e9f105760afff40cb
I20260810 07:52:52.790500 1071 tablet_bootstrap.cc:654] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.892808 1454 consensus_queue.cc:1048] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:52.895380 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 82) took 1002 ms. Trace:
W20260810 07:52:52.896015 574 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 4af5109e338844139444bb7111619195
I20260810 07:52:52.897382 574 leader_election.cc:304] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:52.897145 1263 rpcz_store.cc:276] 0810 07:52:51.896872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.923783 (+ 26911us) service_pool.cc:224] Handling call
0810 07:52:52.895078 (+971295us) ts_tablet_manager.cc:1151] Deleting tablet a1373ebbd3ba4262806bfc17823d25b5
0810 07:52:52.895128 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.895317 (+ 189us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":957529}
I20260810 07:52:52.898308 1454 raft_consensus.cc:2749] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:52.906279 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet a1373ebbd3ba4262806bfc17823d25b5 because the tablet was not found. No further retry: Not found: Tablet not found: a1373ebbd3ba4262806bfc17823d25b5
W20260810 07:52:52.908067 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 because the tablet was not found. No further retry: Not found: Tablet not found: e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:52.909461 1263 tablet_replica.cc:333] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.910275 1263 raft_consensus.cc:2243] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.911619 1263 raft_consensus.cc:2272] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.914531 1263 ts_tablet_manager.cc:1916] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.915180 1047 tablet_bootstrap.cc:654] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.928870 574 leader_election.cc:304] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:52.929803 1454 raft_consensus.cc:2804] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:52.930171 1454 raft_consensus.cc:493] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:52.930497 1454 raft_consensus.cc:3060] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:52.931744 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:52.932487 676 raft_consensus.cc:2393] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:52.910034 1491 raft_consensus.cc:493] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.938184 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:52.941280 574 leader_election.cc:304] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:52.942690 1459 raft_consensus.cc:2804] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:52.943254 1459 raft_consensus.cc:697] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:52.943317 1454 raft_consensus.cc:515] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.944284 1459 consensus_queue.cc:237] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.945820 1454 leader_election.cc:290] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.946094 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:52.947629 573 leader_election.cc:343] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.954509 1047 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:52.954941 1047 ts_tablet_manager.cc:1403] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.095s user 0.006s sys 0.004s
W20260810 07:52:52.957397 792 consensus_peers.cc:597] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:52.957314 1047 raft_consensus.cc:359] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.957926 1047 raft_consensus.cc:385] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:52:52.958120 792 leader_election.cc:343] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:52.958189 1047 raft_consensus.cc:740] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:52.959021 574 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:52.959648 1263 ts_tablet_manager.cc:1929] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:52.960080 1263 log.cc:1199] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/5b76191c5bed41509344376f1810e7aa
I20260810 07:52:52.961298 1390 raft_consensus.cc:2749] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:52:52.964468 32464 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet acc89d15b4d544b7bc8abc12a4c6d3e5 has no consensus state
I20260810 07:52:52.965637 1263 ts_tablet_manager.cc:1950] T 5b76191c5bed41509344376f1810e7aa P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:52.965736 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet f9d3fcf79a104c47a6fd10fc34f7ce08 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:52.968621 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 5b76191c5bed41509344376f1810e7aa (table default.Table_f4269fe23ea14f68b3bbf0770396ffc5 [id=a66d245c7ec84e8080475db270e64746]) successfully deleted
I20260810 07:52:52.969730 1263 tablet_replica.cc:333] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:52.974306 1263 raft_consensus.cc:2243] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:52.981251 1263 pending_rounds.cc:70] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:52.981559 1263 pending_rounds.cc:77] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882929619886080 op_type: NO_OP noop_request { }
I20260810 07:52:52.976347 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_58600ebaa02642eea48fbf663c45afe2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:52.972872 1047 consensus_queue.cc:260] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:52.983094 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58600ebaa02642eea48fbf663c45afe2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:52.983744 1047 ts_tablet_manager.cc:1434] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.029s user 0.006s sys 0.001s
I20260810 07:52:52.990947 1481 raft_consensus.cc:493] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:52.991464 1481 raft_consensus.cc:515] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:52.993963 1481 leader_election.cc:290] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:52.994556 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3a887c70a4ef45ccace031aa6fa907de" } new_table_name: "default.Table_c82ef7c9085a43d8aa2b06a7799dcab9" modify_external_catalogs: true
I20260810 07:52:52.982033 1263 raft_consensus.cc:2889] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:52.996016 1263 raft_consensus.cc:2272] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:52.998307 1263 ts_tablet_manager.cc:1916] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:52.999859 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:53.000713 1047 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:53.010588 792 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:53.017019 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5195424dd98f444caa75e2ca98d09e6d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:53.021389 1047 tablet_bootstrap.cc:654] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:52.937309 1491 raft_consensus.cc:515] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.033608 1490 raft_consensus.cc:2749] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.048525 1491 leader_election.cc:290] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:53.054735 1047 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:53.055204 1047 ts_tablet_manager.cc:1403] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.000s
I20260810 07:52:53.057775 1047 raft_consensus.cc:359] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.058426 1047 raft_consensus.cc:385] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.058702 1047 raft_consensus.cc:740] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.073645 1064 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:53.074081 1064 ts_tablet_manager.cc:1403] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.334s user 0.013s sys 0.006s
I20260810 07:52:53.078336 1047 consensus_queue.cc:260] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.078573 1491 consensus_queue.cc:1048] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:52:53.078112 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46551.9us
I20260810 07:52:53.080597 1047 ts_tablet_manager.cc:1434] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260810 07:52:53.081512 1047 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:53.094693 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:53.095531 896 raft_consensus.cc:3060] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:53.109898 896 raft_consensus.cc:2468] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:53.116487 1263 ts_tablet_manager.cc:1929] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.119383 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e2678ab165486dbbb69c7c183a7fa4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:53.120153 1263 log.cc:1199] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/1ba026e367ea487fb0dc8aef5d5db606
W20260810 07:52:53.130728 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c171d3358e44ae2962ab3c4c377b0f1 has no consensus state
I20260810 07:52:53.140223 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b3adcec668f7473687d358c9b80c10f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:53.144227 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3adcec668f7473687d358c9b80c10f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.155422 1047 tablet_bootstrap.cc:654] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.144861 1263 ts_tablet_manager.cc:1950] T 1ba026e367ea487fb0dc8aef5d5db606 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:53.163002 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 1ba026e367ea487fb0dc8aef5d5db606 (table default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]) successfully deleted
I20260810 07:52:53.110508 1491 raft_consensus.cc:493] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.173637 1263 tablet_replica.cc:333] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:53.178611 1520 tablet.cc:1724] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:53.188359 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f3fa658df8764c2aa73aa960d170e2a3" } new_table_name: "default.Table_c56854332dfd4628a280b1d87bf7a2cb" modify_external_catalogs: true
I20260810 07:52:53.197847 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:52:53.149345 1064 raft_consensus.cc:359] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.204129 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:53.165455 1491 raft_consensus.cc:515] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.206961 1491 leader_election.cc:290] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:53.202803 1064 raft_consensus.cc:385] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.207721 1064 raft_consensus.cc:740] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.208353 1064 consensus_queue.cc:260] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.210461 1064 ts_tablet_manager.cc:1434] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.136s user 0.004s sys 0.001s
I20260810 07:52:53.211016 32460 catalog_manager.cc:6807] ReplaceTablet: tablet 92e2678ab165486dbbb69c7c183a7fa4 deleted and replaced by tablet dd22d1029ff249a0a9e05862293261ff
I20260810 07:52:53.213510 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_5c9348ac7cc14fb1842cb6bd08652cae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:53.216460 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c9348ac7cc14fb1842cb6bd08652cae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.223429 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:53.225138 32464 catalog_manager.cc:6807] ReplaceTablet: tablet f9d3fcf79a104c47a6fd10fc34f7ce08 deleted and replaced by tablet a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:52:53.226004 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0dae8ea1aefd4618b2dddfc4e459f367"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:53.227434 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0dae8ea1aefd4618b2dddfc4e459f367 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.151798 1519 tablet.cc:1724] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:53.240742 1047 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:53.241230 1047 ts_tablet_manager.cc:1403] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.160s user 0.009s sys 0.015s
I20260810 07:52:53.245111 1390 raft_consensus.cc:493] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.245576 1047 raft_consensus.cc:359] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.245746 1390 raft_consensus.cc:515] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.246359 1047 raft_consensus.cc:385] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.246666 1047 raft_consensus.cc:740] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.247434 1047 consensus_queue.cc:260] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.249585 635 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:53.250025 1047 ts_tablet_manager.cc:1434] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260810 07:52:53.250419 1390 leader_election.cc:290] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:53.260849 1047 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:53.273253 1263 raft_consensus.cc:2243] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:53.276842 1064 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:53.279707 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.285076 1263 raft_consensus.cc:2272] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:53.289187 1263 ts_tablet_manager.cc:1916] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:53.296797 1047 tablet_bootstrap.cc:654] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.302454 384 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.280113 309 leader_election.cc:304] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:53.311050 1064 tablet_bootstrap.cc:654] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.307960 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec00a494816409684205c7f4435a079" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:53.316893 1481 raft_consensus.cc:2749] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.318522 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e2678ab165486dbbb69c7c183a7fa4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:53.322865 573 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.323407 1437 ts_tablet_manager.cc:1929] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.323935 1437 log.cc:1199] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:53.326749 1437 ts_tablet_manager.cc:1950] T d397310bcfd2470e95fc4b0654749aa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:53.327994 1071 tablet_bootstrap.cc:492] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:53.328527 1071 ts_tablet_manager.cc:1403] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.564s user 0.003s sys 0.020s
I20260810 07:52:53.330904 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 72) took 1493 ms. Trace:
I20260810 07:52:53.335496 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f89e0d36f8cb41f58f17e017b9172a83" } new_table_name: "default.Table_a448e407c7ec4922ab6bf6b5bb01b6f5" modify_external_catalogs: true
I20260810 07:52:53.335943 1437 rpcz_store.cc:276] 0810 07:52:51.837527 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.837746 (+ 219us) service_pool.cc:224] Handling call
0810 07:52:52.394886 (+557140us) ts_tablet_manager.cc:1151] Deleting tablet d397310bcfd2470e95fc4b0654749aa4
0810 07:52:52.394936 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:52.655110 (+260174us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400100254. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b56d951 00007f9c6b60c742 00007f9c6ba4cd2c 00007f9c6c6b441a 00007f9c6c6e50e0 00007f9c6c6cb64d 00007f9c6c6cb5f9 00007f9c6c6cb589 00007f9c6c6cb551 00007f9c6c6ca77d 00000000005bd8c4
0810 07:52:52.857972 (+202862us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.309126 (+451154us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.330833 (+ 21707us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527165184,"tablet-delete.queue_time_us":556135}
I20260810 07:52:53.331421 1071 raft_consensus.cc:359] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.352411 1071 raft_consensus.cc:385] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.352880 1071 raft_consensus.cc:740] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.353682 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet d397310bcfd2470e95fc4b0654749aa4 (table default.Table_937b2a00ea624497b31e38f446787d9f [id=a77db8548cfa423995f0b6b2eecb2a44]) successfully deleted
I20260810 07:52:53.353636 1071 consensus_queue.cc:260] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.355729 1071 ts_tablet_manager.cc:1434] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260810 07:52:53.357601 1071 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:53.361013 1437 tablet_replica.cc:333] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:53.352689 1263 ts_tablet_manager.cc:1929] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.362346 1263 log.cc:1199] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/acf740de7abb40bfabfc3a7859cc159b
I20260810 07:52:53.363770 1263 ts_tablet_manager.cc:1950] T acf740de7abb40bfabfc3a7859cc159b P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:53.366379 1437 raft_consensus.cc:2243] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:53.366371 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]) successfully deleted
I20260810 07:52:53.366951 1437 raft_consensus.cc:2272] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:53.363817 1071 tablet_bootstrap.cc:654] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.381762 1437 ts_tablet_manager.cc:1916] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:53.381657 1263 tablet_replica.cc:333] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:53.384891 1454 raft_consensus.cc:493] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.385396 1454 raft_consensus.cc:515] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.385486 1263 raft_consensus.cc:2243] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:53.387604 1454 leader_election.cc:290] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:53.388625 1263 raft_consensus.cc:2272] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:53.389839 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:53.391186 573 leader_election.cc:343] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:53.379436 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e2678ab165486dbbb69c7c183a7fa4 already in progress: creating tablet
I20260810 07:52:53.401719 1263 ts_tablet_manager.cc:1916] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:53.404410 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_870a1aff5f614bebbde025da24dedfa8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:53.406621 1047 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:53.407094 1047 ts_tablet_manager.cc:1403] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.156s user 0.016s sys 0.006s
I20260810 07:52:53.410132 1047 raft_consensus.cc:359] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.410929 1047 raft_consensus.cc:385] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.411280 1047 raft_consensus.cc:740] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.412137 1047 consensus_queue.cc:260] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.414654 1047 ts_tablet_manager.cc:1434] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:52:53.415556 1047 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:53.423135 411 raft_consensus.cc:3060] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:53.431437 1047 tablet_bootstrap.cc:654] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:53.439877 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_870a1aff5f614bebbde025da24dedfa8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.471652 1047 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:53.472144 1047 ts_tablet_manager.cc:1403] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.000s
I20260810 07:52:53.474946 1047 raft_consensus.cc:359] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.477124 1071 tablet_bootstrap.cc:492] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:53.477581 1071 ts_tablet_manager.cc:1403] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.001s
I20260810 07:52:53.485675 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d446ae528a0144368b59327e55dd86e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:53.488170 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d446ae528a0144368b59327e55dd86e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.489169 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae4a28e89714c65b7551f36d1dc78b4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:53.493091 574 leader_election.cc:304] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:53.494805 1454 raft_consensus.cc:2804] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:53.495404 1454 raft_consensus.cc:697] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:53.496953 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae4a28e89714c65b7551f36d1dc78b4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:53.497648 676 raft_consensus.cc:2468] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:53.496325 1454 consensus_queue.cc:237] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:53.502750 792 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:53.506475 792 leader_election.cc:343] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.507469 1071 raft_consensus.cc:359] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.508258 1071 raft_consensus.cc:385] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.508556 1071 raft_consensus.cc:740] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:52:53.511026 792 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.514248 1071 consensus_queue.cc:260] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.519336 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:53.520510 792 leader_election.cc:304] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:53.527406 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:53.530397 1481 raft_consensus.cc:493] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.530891 1481 raft_consensus.cc:515] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.533787 1481 leader_election.cc:290] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:53.534601 1071 ts_tablet_manager.cc:1434] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
W20260810 07:52:53.535192 308 leader_election.cc:343] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:53.535655 1071 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:53.475632 1047 raft_consensus.cc:385] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.545089 1047 raft_consensus.cc:740] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.547226 1263 ts_tablet_manager.cc:1929] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:53.547626 1263 log.cc:1199] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:53.548235 1263 ts_tablet_manager.cc:1950] T f9d3fcf79a104c47a6fd10fc34f7ce08 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:53.561085 1047 consensus_queue.cc:260] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.563042 864 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:53.564811 1481 raft_consensus.cc:493] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.565307 1481 raft_consensus.cc:515] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.565907 1047 ts_tablet_manager.cc:1434] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.093s user 0.004s sys 0.004s
I20260810 07:52:53.569095 1481 leader_election.cc:290] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:53.569530 895 raft_consensus.cc:2468] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:53.575812 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
W20260810 07:52:53.577405 308 leader_election.cc:343] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: acf740de7abb40bfabfc3a7859cc159b
W20260810 07:52:53.579664 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 68666.9us
I20260810 07:52:53.580811 792 leader_election.cc:304] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:53.581431 1437 ts_tablet_manager.cc:1929] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.582959 1257 raft_consensus.cc:2749] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.584883 1257 raft_consensus.cc:2804] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:53.585216 1257 raft_consensus.cc:493] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:53.585517 1257 raft_consensus.cc:3060] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:53.585356 574 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 4ec00a494816409684205c7f4435a079
I20260810 07:52:53.585917 574 leader_election.cc:304] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:53.587860 574 leader_election.cc:304] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:53.589558 1390 raft_consensus.cc:2749] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.590525 1390 raft_consensus.cc:2804] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:53.590904 1390 raft_consensus.cc:493] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:53.591253 1390 raft_consensus.cc:3060] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:53.594255 1064 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:53.595068 1064 ts_tablet_manager.cc:1403] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.318s user 0.016s sys 0.018s
I20260810 07:52:53.610445 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf740de7abb40bfabfc3a7859cc159b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:53.620496 1265 ts_tablet_manager.cc:1929] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.621073 1265 log.cc:1199] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/5b76191c5bed41509344376f1810e7aa
I20260810 07:52:53.626025 635 tablet_service.cc:1511] Processing CreateTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.627146 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449359c8e9fa4756ba149f3dce6e0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.629886 864 tablet_service.cc:1511] Processing CreateTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.624600 32468 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:53.628882 1265 ts_tablet_manager.cc:1950] T 5b76191c5bed41509344376f1810e7aa P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:53.637832 32468 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]). Expected version 2 got 0
I20260810 07:52:53.639408 32462 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 4d0b9b2012234cb19d337423b0f316af
I20260810 07:52:53.639816 32462 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 121) took 1187 ms. Trace:
I20260810 07:52:53.640161 32462 rpcz_store.cc:276] 0810 07:52:52.452167 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.689160 (+236993us) service_pool.cc:224] Handling call
0810 07:52:52.750371 (+ 61211us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:52.751261 (+ 890us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:52:52.751571 (+ 310us) catalog_manager.cc:2850] Locking tablets
0810 07:52:52.818103 (+ 66532us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:52:53.046297 (+228194us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:52:53.046586 (+ 289us) catalog_manager.cc:2898] Committing in-memory state
0810 07:52:53.639278 (+592692us) catalog_manager.cc:2902] Aborting table tasks
0810 07:52:53.639748 (+ 470us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.823429 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.823842 (+ 413us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.823866 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:52:52.823877 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:52.824217 (+ 340us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.824272 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.824302 (+ 30us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:52.824557 (+ 255us) tablet.cc:701] Row locks acquired
0810 07:52:52.824590 (+ 33us) write_op.cc:260] PREPARE: finished
0810 07:52:52.824755 (+ 165us) write_op.cc:270] Start()
0810 07:52:52.824854 (+ 99us) write_op.cc:276] Timestamp: P: 1786348372824611 usec, L: 0
0810 07:52:52.824881 (+ 27us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.825452 (+ 571us) log.cc:844] Serialized 533 byte log entry
0810 07:52:53.037171 (+211719us) op_driver.cc:464] REPLICATION: finished
0810 07:52:53.042062 (+ 4891us) write_op.cc:301] APPLY: starting
0810 07:52:53.042116 (+ 54us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:53.042208 (+ 92us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:53.042317 (+ 109us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:53.042570 (+ 253us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:53.042691 (+ 121us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:53.042731 (+ 40us) write_op.cc:312] APPLY: finished
0810 07:52:53.042902 (+ 171us) log.cc:844] Serialized 29 byte log entry
0810 07:52:53.043039 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.043104 (+ 65us) write_op.cc:454] Released schema lock
0810 07:52:53.043133 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4710,"apply.run_cpu_time_us":1276,"apply.run_wall_time_us":1276,"num_ops":2,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":2330,"prepare.run_wall_time_us":2477,"raft.queue_time_us":86,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":212223,"thread_start_us":9383,"threads_started":1}]]}
I20260810 07:52:53.647183 1064 raft_consensus.cc:359] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.647944 1064 raft_consensus.cc:385] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:53.647140 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b041641ad99146e8ac57b26f842b8976"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:53.648212 1064 raft_consensus.cc:740] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:53.649894 1047 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:53.650348 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449359c8e9fa4756ba149f3dce6e0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.657563 1390 raft_consensus.cc:515] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.664266 1491 raft_consensus.cc:493] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.664759 1491 raft_consensus.cc:515] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.665174 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a4eef7144b52473388a18de022fc7566"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:53.666774 1491 leader_election.cc:290] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:53.667894 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4eef7144b52473388a18de022fc7566 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.673882 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:53.675536 792 leader_election.cc:343] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:53.678061 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 90296.9us
I20260810 07:52:53.690879 1047 tablet_bootstrap.cc:654] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.675112 32465 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:53.700181 32465 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98 [id=a49a08c7a7954b87aab561894073bf0d]). Expected version 2 got 0
I20260810 07:52:53.704492 645 tablet_service.cc:1511] Processing CreateTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 (DEFAULT_TABLE table=default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.709178 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf45a33485e9463d89b5ce8affa0bd22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.680240 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 87) took 1653 ms. Trace:
I20260810 07:52:53.714282 1520 tablet.cc:1719] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:53.697728 868 tablet_service.cc:1511] Processing CreateTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 (DEFAULT_TABLE table=default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.717224 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf45a33485e9463d89b5ce8affa0bd22. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:53.719799 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6226a2377864d47b14901df39677127 consensus state has no leader
I20260810 07:52:53.705607 1064 consensus_queue.cc:260] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.723551 1064 ts_tablet_manager.cc:1434] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.128s user 0.005s sys 0.001s
I20260810 07:52:53.725909 1390 leader_election.cc:290] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:53.727438 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:53.728008 892 raft_consensus.cc:3060] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:53.714140 1265 rpcz_store.cc:276] 0810 07:52:52.026910 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.027075 (+ 165us) service_pool.cc:224] Handling call
0810 07:52:52.781899 (+754824us) ts_tablet_manager.cc:1151] Deleting tablet 5b76191c5bed41509344376f1810e7aa
0810 07:52:52.781944 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:53.077482 (+295538us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.620246 (+542764us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.680161 (+ 59915us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":754238}
I20260810 07:52:53.740798 1265 tablet_replica.cc:333] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:53.741722 1265 raft_consensus.cc:2243] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:53.742673 1265 raft_consensus.cc:2272] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:53.746145 32460 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:53.748204 32460 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:52:53.748935 1437 log.cc:1199] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/e44b480f5f3d4f239eae2586c1d2001a
I20260810 07:52:53.749217 1257 raft_consensus.cc:515] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.749974 1437 ts_tablet_manager.cc:1950] T e44b480f5f3d4f239eae2586c1d2001a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:52:53.649861 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b041641ad99146e8ac57b26f842b8976 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.752480 1257 leader_election.cc:290] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:53.753886 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae4a28e89714c65b7551f36d1dc78b4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:53.582551 1551 tablet.cc:1724] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260810 07:52:53.608832 309 leader_election.cc:343] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:53.754511 676 raft_consensus.cc:3060] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:53.754792 309 leader_election.cc:304] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:52:53.755795 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 147009us
I20260810 07:52:53.755964 892 raft_consensus.cc:2468] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:53.757545 309 leader_election.cc:304] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:53.626987 1071 tablet_bootstrap.cc:654] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:53.677841 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe43c4aa983843c68a5dcf85201df941" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:53.698221 377 tablet_service.cc:1511] Processing CreateTablet for tablet a57324aac10d42d191f72e2c2343d4ea (DEFAULT_TABLE table=default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.761152 574 leader_election.cc:304] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:53.761778 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57324aac10d42d191f72e2c2343d4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.762147 1390 raft_consensus.cc:2804] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:53.762657 1390 raft_consensus.cc:697] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:53.763573 1390 consensus_queue.cc:237] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.698630 380 tablet_service.cc:1511] Processing CreateTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 (DEFAULT_TABLE table=default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.765436 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf45a33485e9463d89b5ce8affa0bd22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.625844 384 tablet_service.cc:1511] Processing CreateTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.758330 1481 raft_consensus.cc:2749] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.769016 676 raft_consensus.cc:2468] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:53.769202 1481 raft_consensus.cc:2749] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:53.769415 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449359c8e9fa4756ba149f3dce6e0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.759011 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae4a28e89714c65b7551f36d1dc78b4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:53.770566 792 leader_election.cc:304] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:53.773028 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:53.776039 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 69c05d6e49204eedbb305f66dfa2316c from {username='slave'} at 127.0.0.1:43806
W20260810 07:52:53.779047 573 leader_election.cc:343] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.780968 872 tablet_service.cc:1511] Processing CreateTablet for tablet a57324aac10d42d191f72e2c2343d4ea (DEFAULT_TABLE table=default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:53.781553 1292 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260810 07:52:53.782084 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57324aac10d42d191f72e2c2343d4ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:53.782390 1292 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:52:53.783222 1292 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260810 07:52:53.786062 639 tablet_service.cc:1511] Processing CreateTablet for tablet a57324aac10d42d191f72e2c2343d4ea (DEFAULT_TABLE table=default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.787199 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57324aac10d42d191f72e2c2343d4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.793918 1490 raft_consensus.cc:2804] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:53.800024 1490 raft_consensus.cc:697] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:53.801126 1490 consensus_queue.cc:237] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.802829 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_903e7df2a6cc4ebdb26c728e4c047cfc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:53.807343 1519 tablet.cc:1724] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:53.807966 385 tablet_service.cc:1511] Processing CreateTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 (DEFAULT_TABLE table=default.Table_d6e2c1a2cec54b099ba095449ee1b5a3 [id=3e9b86f943934bd787e7a02164e1efb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.809012 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8779af4159eb48da894e6a92c2c4a6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.811012 638 tablet_service.cc:1511] Processing CreateTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 (DEFAULT_TABLE table=default.Table_d6e2c1a2cec54b099ba095449ee1b5a3 [id=3e9b86f943934bd787e7a02164e1efb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:53.812191 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8779af4159eb48da894e6a92c2c4a6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:53.820222 1491 raft_consensus.cc:493] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.820910 1491 raft_consensus.cc:515] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.824102 1491 leader_election.cc:290] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:53.830487 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:53.831969 792 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.835789 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 71) took 1999 ms. Trace:
I20260810 07:52:53.724584 1064 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:53.805058 32461 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 338e9b0e89d44295973d4003abd36649
I20260810 07:52:53.850659 1552 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:53.805599 32470 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_f89e0d36f8cb41f58f17e017b9172a83 [id=6ed43022a4d74f538afc9f9786e629e1] alter complete (version 2)
I20260810 07:52:53.851153 1552 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:53.820701 32463 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_9f0c5abc8ee0420295dbbbf0f861befa [id=8ebf95685c034249a05157e5d113892d] alter complete (version 2)
I20260810 07:52:53.854977 1552 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:53.824898 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 20a9ab67614949bb93925144c4675f01 consensus state has no leader
I20260810 07:52:53.857481 32469 master_service.cc:960] ReplaceTablet: received request to replace tablet fe43c4aa983843c68a5dcf85201df941 from {username='slave'} at 127.0.0.1:43806
W20260810 07:52:53.870528 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 28) took 2051 ms (client timeout 1000 ms). Trace:
W20260810 07:52:53.871686 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260810 07:52:53.873664 32461 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 122) took 1236 ms. Trace:
I20260810 07:52:53.877213 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_826c5b5c4ab84216af2be3ae4ce22024"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:53.879087 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_826c5b5c4ab84216af2be3ae4ce22024 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.874053 32461 rpcz_store.cc:276] 0810 07:52:52.636838 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.710537 (+ 73699us) service_pool.cc:224] Handling call
0810 07:52:52.756016 (+ 45479us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:52.805500 (+ 49484us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:52:52.805664 (+ 164us) catalog_manager.cc:2850] Locking tablets
0810 07:52:52.806342 (+ 678us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:52:53.047405 (+241063us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:52:53.047622 (+ 217us) catalog_manager.cc:2898] Committing in-memory state
0810 07:52:53.804915 (+757293us) catalog_manager.cc:2902] Aborting table tasks
0810 07:52:53.872547 (+ 67632us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4c00002140. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cd655fc0 00007f30cd666fa9 00007f30cd66768f 00007f30cd62ae7b 00007f30cd6594dc 00007f30cd65999e 00007f30ceff4c1b 00007f30d17b6151 00007f30d175ee60 00007f30d175a460 00007f30d174f690
0810 07:52:53.873597 (+ 1050us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.815396 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.815801 (+ 405us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.815826 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:52:52.815840 (+ 14us) tablet.cc:662] Decoding operations
0810 07:52:52.816134 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.816170 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.816191 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:52.816468 (+ 277us) tablet.cc:701] Row locks acquired
0810 07:52:52.816481 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:52.816594 (+ 113us) write_op.cc:270] Start()
0810 07:52:52.816678 (+ 84us) write_op.cc:276] Timestamp: P: 1786348372816476 usec, L: 0
0810 07:52:52.816692 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.817329 (+ 637us) log.cc:844] Serialized 731 byte log entry
0810 07:52:53.036982 (+219653us) op_driver.cc:464] REPLICATION: finished
0810 07:52:53.041217 (+ 4235us) write_op.cc:301] APPLY: starting
0810 07:52:53.041264 (+ 47us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:53.041363 (+ 99us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:53.041456 (+ 93us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:53.041694 (+ 238us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:53.041850 (+ 156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:53.041895 (+ 45us) write_op.cc:312] APPLY: finished
0810 07:52:53.042072 (+ 177us) log.cc:844] Serialized 29 byte log entry
0810 07:52:53.042210 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.042277 (+ 67us) write_op.cc:454] Released schema lock
0810 07:52:53.042308 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:53.055705 (+ 13397us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.055746 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":46021504,"child_traces":[["op",{"apply.queue_time_us":4111,"apply.run_cpu_time_us":1322,"apply.run_wall_time_us":14440,"num_ops":2,"prepare.queue_time_us":6717,"prepare.run_cpu_time_us":2228,"prepare.run_wall_time_us":2365,"raft.queue_time_us":85,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":220197}]]}
I20260810 07:52:53.882437 32461 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_5b06ee13a0114e059c520a2edd38edf4" } modify_external_catalogs: true
I20260810 07:52:53.883004 32461 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_5b06ee13a0114e059c520a2edd38edf4" } modify_external_catalogs: true
I20260810 07:52:53.885010 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:53.886369 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b76191c5bed41509344376f1810e7aa with delete_type TABLET_DATA_DELETED (replaced by tablet 4d0b9b2012234cb19d337423b0f316af) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:53.893755 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_903e7df2a6cc4ebdb26c728e4c047cfc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:53.897004 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:53.897800 1263 tablet_replica.cc:333] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:53.898569 1263 raft_consensus.cc:2243] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:53.899269 1263 pending_rounds.cc:70] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:53.899492 1263 pending_rounds.cc:77] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882935594295296 op_type: NO_OP noop_request { }
I20260810 07:52:53.899865 1263 raft_consensus.cc:2889] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:53.900302 1263 raft_consensus.cc:2272] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:53.904366 1263 ts_tablet_manager.cc:1916] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:53.924840 1047 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:53.925313 1047 ts_tablet_manager.cc:1403] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.276s user 0.005s sys 0.007s
I20260810 07:52:53.929996 382 tablet_service.cc:1558] Processing DeleteTablet for tablet a1373ebbd3ba4262806bfc17823d25b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c075fdf1055477db9f2d56324f12a8a) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.930299 376 tablet_service.cc:1558] Processing DeleteTablet for tablet f353e1e07485400a964f8e05b0691cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.931028 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.966238 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:53.967638 1263 ts_tablet_manager.cc:1929] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:53.968039 1263 log.cc:1199] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/338e9b0e89d44295973d4003abd36649
I20260810 07:52:53.968816 1263 ts_tablet_manager.cc:1950] T 338e9b0e89d44295973d4003abd36649 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:53.978150 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:52:53.871572 32463 rpcz_store.cc:269] 0810 07:52:51.819280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.907766 (+ 88486us) service_pool.cc:224] Handling call
0810 07:52:53.869647 (+1961881us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4c00002140. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cd655fc0 00007f30cd666fa9 00007f30cd66768f 00007f30cd62ae7b 00007f30cd6594dc 00007f30cd65999e 00007f30ceff4c1b 00007f30d17b6151 00007f30d175ee60 00007f30d1764f3b 00007f30d186b53e
0810 07:52:53.870454 (+ 807us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.295913 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.296371 (+ 458us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.296398 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:52:52.296413 (+ 15us) tablet.cc:662] Decoding operations
0810 07:52:52.296783 (+ 370us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.296842 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.296869 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:52.297041 (+ 172us) tablet.cc:701] Row locks acquired
0810 07:52:52.297055 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:52.297199 (+ 144us) write_op.cc:270] Start()
0810 07:52:52.297299 (+ 100us) write_op.cc:276] Timestamp: P: 1786348372297124 usec, L: 0
0810 07:52:52.297316 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.334825 (+ 37509us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b5400030aa0. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30d07a0964 00007f30cf97298b 00007f30cf9726da 00007f30cf9f6d45 00007f30cf9f71e9 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1
0810 07:52:52.335461 (+ 636us) log.cc:844] Serialized 356 byte log entry
0810 07:52:52.911943 (+576482us) op_driver.cc:464] REPLICATION: finished
0810 07:52:52.912838 (+ 895us) write_op.cc:301] APPLY: starting
0810 07:52:52.912946 (+ 108us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:52.913093 (+ 147us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:52.913112 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:52.913357 (+ 245us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:52.913543 (+ 186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:52.913607 (+ 64us) write_op.cc:312] APPLY: finished
0810 07:52:52.913841 (+ 234us) log.cc:844] Serialized 23 byte log entry
0810 07:52:52.914030 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.914122 (+ 92us) write_op.cc:454] Released schema lock
0810 07:52:52.914155 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:52.914451 (+ 296us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:52.914492 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":27737344,"child_traces":[["op",{"apply.queue_time_us":625,"apply.run_cpu_time_us":1547,"apply.run_wall_time_us":1547,"num_ops":1,"prepare.queue_time_us":8905,"prepare.run_cpu_time_us":3927,"prepare.run_wall_time_us":41091,"replication_time_us":614486,"spinlock_wait_cycles":59820416}]]}
W20260810 07:52:53.895905 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 1ba026e367ea487fb0dc8aef5d5db606 because the tablet was not found. No further retry: Not found: Tablet not found: 1ba026e367ea487fb0dc8aef5d5db606
W20260810 07:52:53.874512 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 18) took 1723 ms (client timeout 1000 ms). Trace:
W20260810 07:52:53.989822 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 93951.3us
I20260810 07:52:53.989962 1265 ts_tablet_manager.cc:1916] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:53.990414 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 5b76191c5bed41509344376f1810e7aa because the tablet was not found. No further retry: Not found: Tablet not found: 5b76191c5bed41509344376f1810e7aa
I20260810 07:52:53.991241 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 338e9b0e89d44295973d4003abd36649 (table default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]) successfully deleted
W20260810 07:52:53.897975 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 20) took 1456 ms (client timeout 1000 ms). Trace:
I20260810 07:52:53.918051 32465 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5] alter complete (version 2)
I20260810 07:52:53.933945 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet fe43c4aa983843c68a5dcf85201df941
W20260810 07:52:53.994613 574 leader_election.cc:343] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.001322 1519 tablet.cc:1719] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Skipping requested alter to schema version 2, tablet already version 2
I20260810 07:52:54.002355 639 tablet_service.cc:1511] Processing CreateTablet for tablet e38ab80cd5c441f3ba20bf00577b223c (DEFAULT_TABLE table=default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.003417 1064 tablet_bootstrap.cc:654] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.003669 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ab80cd5c441f3ba20bf00577b223c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:53.989741 32470 rpcz_store.cc:269] 0810 07:52:52.158121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.158308 (+ 187us) service_pool.cc:224] Handling call
0810 07:52:53.867981 (+1709673us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4c00002140. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cd655fc0 00007f30cd666fa9 00007f30cd66768f 00007f30cd62ae7b 00007f30cd6594dc 00007f30cd65999e 00007f30ceff4ddb 00007f30d17b9f86 00007f30d175ee60 00007f30d1764f3b 00007f30d186b53e
0810 07:52:53.874444 (+ 6463us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:52.705197 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:52.705850 (+ 653us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:52.706135 (+ 285us) write_op.cc:435] Acquired schema lock
0810 07:52:52.706156 (+ 21us) tablet.cc:662] Decoding operations
0810 07:52:52.726184 (+ 20028us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:52.726239 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:52.726265 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:52.726516 (+ 251us) tablet.cc:701] Row locks acquired
0810 07:52:52.726533 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:52:52.726666 (+ 133us) write_op.cc:270] Start()
0810 07:52:52.726754 (+ 88us) write_op.cc:276] Timestamp: P: 1786348372726551 usec, L: 0
0810 07:52:52.726773 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:52:52.727514 (+ 741us) log.cc:844] Serialized 533 byte log entry
0810 07:52:53.036680 (+309166us) op_driver.cc:464] REPLICATION: finished
0810 07:52:53.037734 (+ 1054us) write_op.cc:301] APPLY: starting
0810 07:52:53.037796 (+ 62us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:53.037886 (+ 90us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:53.037980 (+ 94us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:53.038167 (+ 187us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:53.038296 (+ 129us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:53.038336 (+ 40us) write_op.cc:312] APPLY: finished
0810 07:52:53.038530 (+ 194us) log.cc:844] Serialized 23 byte log entry
0810 07:52:53.038692 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.038759 (+ 67us) write_op.cc:454] Released schema lock
0810 07:52:53.038793 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:53.040960 (+ 2167us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.040983 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0810 07:52:53.207775 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:53.208270 (+ 495us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:53.208294 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:52:53.208306 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:53.225642 (+ 17336us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:53.225685 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:53.225706 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:53.225844 (+ 138us) tablet.cc:701] Row locks acquired
0810 07:52:53.225856 (+ 12us) write_op.cc:260] PREPARE: finished
0810 07:52:53.225961 (+ 105us) write_op.cc:270] Start()
0810 07:52:53.226044 (+ 83us) write_op.cc:276] Timestamp: P: 1786348373225899 usec, L: 0
0810 07:52:53.226059 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:53.226512 (+ 453us) log.cc:844] Serialized 356 byte log entry
0810 07:52:53.257173 (+ 30661us) op_driver.cc:464] REPLICATION: finished
0810 07:52:53.258624 (+ 1451us) write_op.cc:301] APPLY: starting
0810 07:52:53.258689 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:53.258788 (+ 99us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:53.258872 (+ 84us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:53.259080 (+ 208us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:53.259213 (+ 133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:53.259260 (+ 47us) write_op.cc:312] APPLY: finished
0810 07:52:53.259469 (+ 209us) log.cc:844] Serialized 23 byte log entry
0810 07:52:53.260127 (+ 658us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.260192 (+ 65us) write_op.cc:454] Released schema lock
0810 07:52:53.260227 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:53.260700 (+ 473us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.260773 (+ 73us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":24042752,"child_traces":[["op",{"apply.queue_time_us":849,"apply.run_cpu_time_us":1275,"apply.run_wall_time_us":3186,"num_ops":1,"prepare.queue_time_us":216733,"prepare.run_cpu_time_us":2746,"prepare.run_wall_time_us":22857,"raft.queue_time_us":96,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":309813}],["op",{"apply.queue_time_us":1248,"apply.run_cpu_time_us":1387,"apply.run_wall_time_us":1994,"num_ops":1,"prepare.queue_time_us":148507,"prepare.run_cpu_time_us":1883,"prepare.run_wall_time_us":20419,"replication_time_us":30941}]]}
I20260810 07:52:54.014155 644 tablet_service.cc:1511] Processing CreateTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.014245 646 tablet_service.cc:1511] Processing CreateTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 (DEFAULT_TABLE table=default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.015306 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1faed468bfa4d1ba5d92c0b9b194781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.015878 642 tablet_service.cc:1558] Processing DeleteTablet for tablet fe43c4aa983843c68a5dcf85201df941 with delete_type TABLET_DATA_DELETED (replaced by tablet 7eeabaae541548afab4a3ff5ab388035) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.017270 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c171d3358e44ae2962ab3c4c377b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.021813 1390 raft_consensus.cc:493] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:54.022250 1390 raft_consensus.cc:515] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.023502 866 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:52:54.024504 1292 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:52:54.024986 1292 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:52:54.025404 1292 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:52:54.025786 1292 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:52:54.025830 1390 leader_election.cc:290] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:54.025772 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:54.026494 896 raft_consensus.cc:2468] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:54.027659 574 leader_election.cc:304] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:54.029018 645 tablet_service.cc:1558] Processing DeleteTablet for tablet acf740de7abb40bfabfc3a7859cc159b with delete_type TABLET_DATA_DELETED (replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.033130 1263 tablet_replica.cc:333] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:52:54.034422 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 276951us
I20260810 07:52:54.036316 866 tablet_service.cc:1511] Processing CreateTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 (DEFAULT_TABLE table=default.Table_d6e2c1a2cec54b099ba095449ee1b5a3 [id=3e9b86f943934bd787e7a02164e1efb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.036978 645 tablet_service.cc:1511] Processing CreateTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (DEFAULT_TABLE table=default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.038533 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8779af4159eb48da894e6a92c2c4a6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.039944 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc89d15b4d544b7bc8abc12a4c6d3e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:54.008153 32468 rpcz_store.cc:269] 0810 07:52:52.441775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.473681 (+ 31906us) service_pool.cc:224] Handling call
0810 07:52:53.897903 (+1424222us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":9}
I20260810 07:52:54.010535 32469 catalog_manager.cc:6807] ReplaceTablet: tablet fe43c4aa983843c68a5dcf85201df941 deleted and replaced by tablet 7eeabaae541548afab4a3ff5ab388035
W20260810 07:52:54.044277 792 leader_election.cc:343] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:53.943761 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 69c05d6e49204eedbb305f66dfa2316c
I20260810 07:52:54.047829 1390 raft_consensus.cc:2804] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:54.048247 1390 raft_consensus.cc:493] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:54.048592 1390 raft_consensus.cc:3060] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:54.050877 872 tablet_service.cc:1511] Processing CreateTablet for tablet e38ab80cd5c441f3ba20bf00577b223c (DEFAULT_TABLE table=default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.051396 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_ec03572c5d844816b4b09051f198f554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:54.057991 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.060992 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:54.061431 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 69c05d6e49204eedbb305f66dfa2316c with delete_type TABLET_DATA_DELETED (replaced by tablet 8718f998623e4f1c8b7b8be815e184a3) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:54.057389 792 leader_election.cc:343] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.073122 792 leader_election.cc:304] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:54.053429 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_9a4d68bb16a74d4786e2176453635dc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:54.033633 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
W20260810 07:52:54.075840 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec03572c5d844816b4b09051f198f554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:54.076951 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_a448e407c7ec4922ab6bf6b5bb01b6f5 [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 3 got 2
W20260810 07:52:54.077152 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a4d68bb16a74d4786e2176453635dc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:54.077618 32464 catalog_manager.cc:6807] ReplaceTablet: tablet 69c05d6e49204eedbb305f66dfa2316c deleted and replaced by tablet 8718f998623e4f1c8b7b8be815e184a3
W20260810 07:52:54.077740 792 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.078125 792 leader_election.cc:304] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:54.079351 1491 raft_consensus.cc:2749] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.079864 1491 raft_consensus.cc:2749] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.062027 864 tablet_service.cc:1511] Processing CreateTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.081265 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1faed468bfa4d1ba5d92c0b9b194781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.064692 869 tablet_service.cc:1558] Processing DeleteTablet for tablet fe43c4aa983843c68a5dcf85201df941 with delete_type TABLET_DATA_DELETED (replaced by tablet 7eeabaae541548afab4a3ff5ab388035) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:54.064584 862 tablet_service.cc:1511] Processing CreateTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (DEFAULT_TABLE table=default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.082389 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc89d15b4d544b7bc8abc12a4c6d3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.063839 871 tablet_service.cc:1511] Processing CreateTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 (DEFAULT_TABLE table=default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.088075 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:54.089120 382 tablet_service.cc:1511] Processing CreateTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (DEFAULT_TABLE table=default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.089771 380 tablet_service.cc:1511] Processing CreateTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.090169 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc89d15b4d544b7bc8abc12a4c6d3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.090934 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1faed468bfa4d1ba5d92c0b9b194781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.094208 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ab80cd5c441f3ba20bf00577b223c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.094820 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c171d3358e44ae2962ab3c4c377b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.066768 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_29c70477feb94d599ff0c03f584bc04b" } new_table_name: "default.Table_db3c7e150df447dab749a578605e8d4c" modify_external_catalogs: true
I20260810 07:52:54.099579 1520 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:52:54.101744 376 tablet_service.cc:1558] Processing DeleteTablet for tablet fe43c4aa983843c68a5dcf85201df941 with delete_type TABLET_DATA_DELETED (replaced by tablet 7eeabaae541548afab4a3ff5ab388035) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.102018 373 tablet_service.cc:1558] Processing DeleteTablet for tablet 69c05d6e49204eedbb305f66dfa2316c with delete_type TABLET_DATA_DELETED (replaced by tablet 8718f998623e4f1c8b7b8be815e184a3) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:53.836266 1437 rpcz_store.cc:276] 0810 07:52:51.836762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.842195 (+ 5433us) service_pool.cc:224] Handling call
0810 07:52:53.360779 (+1518584us) ts_tablet_manager.cc:1151] Deleting tablet e44b480f5f3d4f239eae2586c1d2001a
0810 07:52:53.360839 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:53.483960 (+123121us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.580606 (+ 96646us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:53.835721 (+255115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1517836}
I20260810 07:52:54.111558 1437 tablet_replica.cc:333] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:54.112444 1437 raft_consensus.cc:2243] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.113054 1437 raft_consensus.cc:2272] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.115207 1437 ts_tablet_manager.cc:1916] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:54.121006 573 leader_election.cc:343] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.106042 375 tablet_service.cc:1511] Processing CreateTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 (DEFAULT_TABLE table=default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.126917 860 tablet_service.cc:1558] Processing DeleteTablet for tablet 69c05d6e49204eedbb305f66dfa2316c with delete_type TABLET_DATA_DELETED (replaced by tablet 8718f998623e4f1c8b7b8be815e184a3) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:54.131309 1265 ts_tablet_manager.cc:1929] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:54.131779 1265 log.cc:1199] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/acf740de7abb40bfabfc3a7859cc159b
I20260810 07:52:54.132608 1265 ts_tablet_manager.cc:1950] T acf740de7abb40bfabfc3a7859cc159b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:53.964650 1047 raft_consensus.cc:359] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.153853 1047 raft_consensus.cc:385] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.154208 1047 raft_consensus.cc:740] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.161371 860 tablet_service.cc:1511] Processing CreateTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.162945 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afbac73bbeb44af6b39ab23c5cf79eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.168852 385 tablet_service.cc:1511] Processing CreateTablet for tablet e38ab80cd5c441f3ba20bf00577b223c (DEFAULT_TABLE table=default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.170265 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ab80cd5c441f3ba20bf00577b223c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.181962 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 91) took 1709 ms. Trace:
I20260810 07:52:54.182343 1265 rpcz_store.cc:276] 0810 07:52:52.472263 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.472406 (+ 143us) service_pool.cc:224] Handling call
0810 07:52:53.740562 (+1268156us) ts_tablet_manager.cc:1151] Deleting tablet acf740de7abb40bfabfc3a7859cc159b
0810 07:52:53.740639 (+ 77us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.044891 (+304252us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.131176 (+ 86285us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.181884 (+ 50708us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1219260}
I20260810 07:52:54.183568 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]) successfully deleted
I20260810 07:52:54.149390 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet e44b480f5f3d4f239eae2586c1d2001a (table default.Table_b21be299a7664348b1a5fb51d9f031ed [id=cac3e96f36a8456d93aac70cbc84a311]) successfully deleted
I20260810 07:52:54.189352 1292 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38193,127.31.139.125:46845,127.31.139.124:36273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.139.126
--webserver_port=38367
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:52:54.191717 1292 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:52:54.194936 1292 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260810 07:52:54.218755 1265 tablet_replica.cc:333] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.219897 1265 raft_consensus.cc:2243] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.272879 1265 pending_rounds.cc:70] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Trying to abort 1 pending ops.
I20260810 07:52:54.273312 1265 pending_rounds.cc:77] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882929619886080 op_type: NO_OP noop_request { }
I20260810 07:52:54.273782 1265 raft_consensus.cc:2889] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:54.274264 1265 raft_consensus.cc:2272] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.277884 639 tablet_service.cc:1511] Processing CreateTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.279105 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afbac73bbeb44af6b39ab23c5cf79eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.188665 382 tablet_service.cc:1511] Processing CreateTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed (DEFAULT_TABLE table=default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.290784 1265 ts_tablet_manager.cc:1916] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.196924 1071 tablet_bootstrap.cc:492] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:54.291661 1071 ts_tablet_manager.cc:1403] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.756s user 0.009s sys 0.010s
I20260810 07:52:54.236268 1551 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:52:54.294422 1263 raft_consensus.cc:2243] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
W20260810 07:52:54.294581 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 173587us
I20260810 07:52:54.302167 1071 raft_consensus.cc:359] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.302989 1071 raft_consensus.cc:385] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.303296 1071 raft_consensus.cc:740] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.304603 1071 consensus_queue.cc:260] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.305263 1263 pending_rounds.cc:70] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:54.314157 1263 pending_rounds.cc:77] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882938975887360 op_type: NO_OP noop_request { }
I20260810 07:52:54.313982 1519 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:52:54.314545 1263 raft_consensus.cc:2889] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:54.314896 1263 raft_consensus.cc:2272] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:54.313707 573 leader_election.cc:343] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.320983 573 leader_election.cc:304] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:54.323673 1552 raft_consensus.cc:2749] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.326459 1263 ts_tablet_manager.cc:1916] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.331768 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afbac73bbeb44af6b39ab23c5cf79eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.332999 1064 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:54.333500 1064 ts_tablet_manager.cc:1403] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.609s user 0.009s sys 0.033s
I20260810 07:52:54.336148 1064 raft_consensus.cc:359] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.336990 1064 raft_consensus.cc:385] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.337284 1064 raft_consensus.cc:740] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.337787 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c171d3358e44ae2962ab3c4c377b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.338076 1064 consensus_queue.cc:260] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.340276 1064 ts_tablet_manager.cc:1434] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:52:54.351398 1064 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:54.355540 1071 ts_tablet_manager.cc:1434] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.064s user 0.003s sys 0.004s
I20260810 07:52:54.366446 1064 tablet_bootstrap.cc:654] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.378593 1580 tablet.cc:1719] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Skipping requested alter to schema version 2, tablet already version 2
W20260810 07:52:54.382925 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 104061us
I20260810 07:52:54.324441 1047 consensus_queue.cc:260] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.388434 1064 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:54.388986 1064 ts_tablet_manager.cc:1403] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.004s
I20260810 07:52:54.391649 1064 raft_consensus.cc:359] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.392347 1064 raft_consensus.cc:385] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.392659 1064 raft_consensus.cc:740] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.391827 32460 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98 [id=a49a08c7a7954b87aab561894073bf0d] alter complete (version 2)
I20260810 07:52:54.393579 1064 consensus_queue.cc:260] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.405784 1490 raft_consensus.cc:493] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:52:54.406454 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 29) took 1512 ms (client timeout 1000 ms). Trace:
I20260810 07:52:54.406441 1490 raft_consensus.cc:515] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.411120 1490 leader_election.cc:290] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:54.417441 1064 ts_tablet_manager.cc:1434] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260810 07:52:54.421815 32464 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_9f0c5abc8ee0420295dbbbf0f861befa" } new_table_name: "default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0" modify_external_catalogs: true
I20260810 07:52:54.430032 32463 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_746adafe126742f79e73baaf7b1fcfa7" } modify_external_catalogs: true
I20260810 07:52:54.431659 1064 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:54.437286 1265 ts_tablet_manager.cc:1929] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:54.437698 1265 log.cc:1199] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/1ba026e367ea487fb0dc8aef5d5db606
I20260810 07:52:54.438652 1265 ts_tablet_manager.cc:1950] T 1ba026e367ea487fb0dc8aef5d5db606 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:54.441224 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 90) took 2007 ms. Trace:
I20260810 07:52:54.441597 1265 rpcz_store.cc:276] 0810 07:52:52.433560 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.453125 (+ 19565us) service_pool.cc:224] Handling call
0810 07:52:54.191723 (+1738598us) ts_tablet_manager.cc:1151] Deleting tablet 1ba026e367ea487fb0dc8aef5d5db606
0810 07:52:54.218600 (+ 26877us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.357313 (+138713us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.437155 (+ 79842us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.441152 (+ 3997us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1644497}
I20260810 07:52:54.442752 1265 tablet_replica.cc:333] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.443753 1265 raft_consensus.cc:2243] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.444326 1265 raft_consensus.cc:2272] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.400696 1071 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:54.454947 32460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 21) took 1000 ms (client timeout 1000 ms). Trace:
I20260810 07:52:54.455426 1265 ts_tablet_manager.cc:1916] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:54.406761 32465 rpcz_store.cc:269] 0810 07:52:52.894431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.933391 (+ 38960us) service_pool.cc:224] Handling call
0810 07:52:52.977453 (+ 44062us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:54.406378 (+1428925us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:53.858425 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:53.858805 (+ 380us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:53.858828 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:52:53.858839 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:53.859088 (+ 249us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:53.859123 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:53.859144 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:53.859303 (+ 159us) tablet.cc:701] Row locks acquired
0810 07:52:53.859316 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:53.859423 (+ 107us) write_op.cc:270] Start()
0810 07:52:53.859520 (+ 97us) write_op.cc:276] Timestamp: P: 1786348373859345 usec, L: 0
0810 07:52:53.859535 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:53.859988 (+ 453us) log.cc:844] Serialized 356 byte log entry
0810 07:52:53.914106 (+ 54118us) op_driver.cc:464] REPLICATION: finished
0810 07:52:53.914843 (+ 737us) write_op.cc:301] APPLY: starting
0810 07:52:53.914899 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:53.914988 (+ 89us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:53.915082 (+ 94us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:53.915256 (+ 174us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:53.915379 (+ 123us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:53.915422 (+ 43us) write_op.cc:312] APPLY: finished
0810 07:52:53.915596 (+ 174us) log.cc:844] Serialized 23 byte log entry
0810 07:52:53.915745 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.915812 (+ 67us) write_op.cc:454] Released schema lock
0810 07:52:53.915841 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:53.916074 (+ 233us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:53.916096 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":95816960,"child_traces":[["op",{"apply.queue_time_us":566,"apply.run_cpu_time_us":1195,"apply.run_wall_time_us":1196,"num_ops":1,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":1749,"prepare.run_wall_time_us":1750,"replication_time_us":54481}]]}
I20260810 07:52:54.460800 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 1ba026e367ea487fb0dc8aef5d5db606 (table default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]) successfully deleted
I20260810 07:52:54.469586 1490 raft_consensus.cc:493] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:54.470105 1490 raft_consensus.cc:515] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.472151 1490 leader_election.cc:290] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:54.472123 1064 tablet_bootstrap.cc:654] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.473843 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:54.476377 792 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:54.481204 1599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:52:54.481161 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:54.484889 792 leader_election.cc:343] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.485034 1265 ts_tablet_manager.cc:1929] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:54.485402 1265 log.cc:1199] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:52:54.486022 1265 ts_tablet_manager.cc:1950] T 92e2678ab165486dbbb69c7c183a7fa4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:54.488415 1265 tablet_replica.cc:333] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.501173 1265 raft_consensus.cc:2243] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.501890 1265 raft_consensus.cc:2272] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.504202 380 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f8941c49c0a4fba9f7de2aabb790901) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.506817 1265 ts_tablet_manager.cc:1916] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.517556 1263 ts_tablet_manager.cc:1929] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:54.518177 1263 log.cc:1199] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/fe43c4aa983843c68a5dcf85201df941
I20260810 07:52:54.520987 1263 ts_tablet_manager.cc:1950] T fe43c4aa983843c68a5dcf85201df941 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:54.522428 1047 ts_tablet_manager.cc:1434] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.597s user 0.008s sys 0.000s
I20260810 07:52:54.524655 1555 raft_consensus.cc:493] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:54.525244 1555 raft_consensus.cc:515] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.528775 1555 leader_election.cc:290] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:54.543527 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c865327885b64e23882fd2d6a58afd0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:54.545945 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c865327885b64e23882fd2d6a58afd0d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:54.558208 309 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:54.559984 868 tablet_service.cc:1511] Processing CreateTablet for tablet b6226a2377864d47b14901df39677127 (DEFAULT_TABLE table=default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.561117 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6226a2377864d47b14901df39677127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.561668 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.562372 384 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fcae5b2e5448bf89b1c0587f46ad40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.563727 384 tablet_service.cc:1558] Processing DeleteTablet for tablet d397310bcfd2470e95fc4b0654749aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:47 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.586488 1064 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:54.586925 1064 ts_tablet_manager.cc:1403] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.156s user 0.010s sys 0.028s
I20260810 07:52:54.589401 1064 raft_consensus.cc:359] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.590112 1064 raft_consensus.cc:385] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.590371 1064 raft_consensus.cc:740] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.591086 1064 consensus_queue.cc:260] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.560465 1265 ts_tablet_manager.cc:1929] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:54.596534 384 tablet_service.cc:1511] Processing CreateTablet for tablet b6226a2377864d47b14901df39677127 (DEFAULT_TABLE table=default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.597008 1265 log.cc:1199] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:54.597705 1265 ts_tablet_manager.cc:1950] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:54.597749 1064 ts_tablet_manager.cc:1434] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.011s user 0.006s sys 0.001s
I20260810 07:52:54.601590 1064 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:54.602737 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 93) took 1086 ms. Trace:
I20260810 07:52:54.603101 1265 rpcz_store.cc:276] 0810 07:52:53.516530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.546260 (+ 29730us) service_pool.cc:224] Handling call
0810 07:52:54.488256 (+941996us) ts_tablet_manager.cc:1151] Deleting tablet f9d3fcf79a104c47a6fd10fc34f7ce08
0810 07:52:54.488301 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.514642 (+ 26341us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.560312 (+ 45670us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.602680 (+ 42368us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":924483}
I20260810 07:52:54.609334 1265 tablet_replica.cc:333] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.610421 1265 raft_consensus.cc:2243] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.606889 1064 tablet_bootstrap.cc:654] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.617096 1265 raft_consensus.cc:2272] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.619143 1265 ts_tablet_manager.cc:1916] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.549315 32461 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 6e63a4f9a3ba4ed4974345bb1aad8e23
I20260810 07:52:54.552206 32463 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_746adafe126742f79e73baaf7b1fcfa7" } modify_external_catalogs: true
I20260810 07:52:54.555945 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 92e2678ab165486dbbb69c7c183a7fa4 (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]) successfully deleted
W20260810 07:52:54.626947 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 71022.9us
W20260810 07:52:54.455336 32460 rpcz_store.cc:269] 0810 07:52:53.454501 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.454731 (+ 230us) service_pool.cc:224] Handling call
0810 07:52:54.454851 (+1000120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:53.989427 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:54.011020 (+ 21593us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:54.011047 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:52:54.011059 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:54.011329 (+ 270us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:54.011369 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:54.011392 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:54.011535 (+ 143us) tablet.cc:701] Row locks acquired
0810 07:52:54.011548 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:54.011666 (+ 118us) write_op.cc:270] Start()
0810 07:52:54.011750 (+ 84us) write_op.cc:276] Timestamp: P: 1786348374011623 usec, L: 0
0810 07:52:54.011765 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:54.012295 (+ 530us) log.cc:844] Serialized 356 byte log entry
0810 07:52:54.387712 (+375417us) op_driver.cc:464] REPLICATION: finished
0810 07:52:54.388319 (+ 607us) write_op.cc:301] APPLY: starting
0810 07:52:54.388385 (+ 66us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:54.388471 (+ 86us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:54.388577 (+ 106us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:54.388831 (+ 254us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:54.388961 (+ 130us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:54.389009 (+ 48us) write_op.cc:312] APPLY: finished
0810 07:52:54.389197 (+ 188us) log.cc:844] Serialized 23 byte log entry
0810 07:52:54.389352 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:54.389412 (+ 60us) write_op.cc:454] Released schema lock
0810 07:52:54.389444 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:54.389660 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:54.389685 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14698,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":436,"apply.run_cpu_time_us":1311,"apply.run_wall_time_us":1311,"num_ops":1,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":2052,"prepare.run_wall_time_us":59792,"raft.queue_time_us":108,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":17,"replication_time_us":375844}]]}
W20260810 07:52:54.631147 644 consensus_meta.cc:287] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Time spent flushing consensus metadata: real 0.573s user 0.006s sys 0.002s
W20260810 07:52:54.522749 1047 ts_tablet_manager.cc:1467] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Tablet startup took 597ms
I20260810 07:52:54.630915 1064 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:54.634747 1064 ts_tablet_manager.cc:1403] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.001s
I20260810 07:52:54.637229 1064 raft_consensus.cc:359] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.638917 1064 raft_consensus.cc:385] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.639528 1064 raft_consensus.cc:740] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.639550 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
I20260810 07:52:54.640524 1064 consensus_queue.cc:260] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.641352 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6226a2377864d47b14901df39677127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.647082 1064 ts_tablet_manager.cc:1434] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260810 07:52:54.647982 1064 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:54.650635 1071 tablet_bootstrap.cc:654] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.471155 1390 raft_consensus.cc:515] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.663875 1263 tablet_replica.cc:333] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:54.666862 1265 ts_tablet_manager.cc:1929] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:54.667325 1265 log.cc:1199] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/338e9b0e89d44295973d4003abd36649
I20260810 07:52:54.667840 1064 tablet_bootstrap.cc:654] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.668079 1265 ts_tablet_manager.cc:1950] T 338e9b0e89d44295973d4003abd36649 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:54.675072 1265 tablet_replica.cc:333] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.675863 1265 raft_consensus.cc:2243] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.675908 1390 leader_election.cc:290] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:54.678488 1265 raft_consensus.cc:2272] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.680241 1265 ts_tablet_manager.cc:1916] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.681119 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:54.683918 380 tablet_service.cc:1558] Processing DeleteTablet for tablet c3675c5bae7a42769223ef5b2fc433b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2c171d3358e44ae2962ab3c4c377b0f1) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:54.633045 1047 ts_tablet_manager.cc:1469] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Trace:
0810 07:52:51.578946 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:51.580087 (+ 1141us) service_pool.cc:224] Handling call
0810 07:52:51.641291 (+ 61204us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b5800000f98. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22b54e6981 00007f22b5626521 00007f22b56263d2 00007f22be56afb9 00007f22bd7ac640 00007f22bd891641 00007f22bd86e8d3 00007f22bd81d210 00007f22bbda9bf4 00007f22bbda9b82 00007f22bbda9ab1
0810 07:52:51.641432 (+ 141us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:51.641533 (+ 101us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:51.822772 (+181239us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:52.036848 (+214076us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:52.085319 (+ 48471us) inbound_call.cc:177] Queueing success response
0810 07:52:53.649559 (+1564240us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0810 07:52:53.925600 (+276041us) ts_tablet_manager.cc:1435] Starting tablet replica
0810 07:52:53.926119 (+ 519us) tablet_replica.cc:261] Starting instrumentation
0810 07:52:53.926556 (+ 437us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":87166,"raft.queue_time_us":87549,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":43976,"spinlock_wait_cycles":19248384,"tablet-open.queue_time_us":1612362}
I20260810 07:52:54.688578 1047 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:54.689500 1437 ts_tablet_manager.cc:1929] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:54.689746 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 338e9b0e89d44295973d4003abd36649 (table default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]) successfully deleted
I20260810 07:52:54.689879 1437 log.cc:1199] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/5b76191c5bed41509344376f1810e7aa
I20260810 07:52:54.690677 1437 ts_tablet_manager.cc:1950] T 5b76191c5bed41509344376f1810e7aa P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:54.670888 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:54.702461 792 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.702868 792 leader_election.cc:304] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:54.708926 1490 raft_consensus.cc:2749] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.711295 1263 raft_consensus.cc:2243] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:54.712074 1263 raft_consensus.cc:2272] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.714100 1263 ts_tablet_manager.cc:1916] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:54.720669 792 leader_election.cc:343] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.721113 792 leader_election.cc:304] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:54.722255 1490 raft_consensus.cc:2749] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.724295 1047 tablet_bootstrap.cc:654] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.724380 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:54.724880 896 raft_consensus.cc:3060] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:54.730852 1490 consensus_queue.cc:1048] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:54.735368 896 raft_consensus.cc:2468] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:54.741516 574 leader_election.cc:304] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:54.742697 1600 consensus_queue.cc:1048] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:54.743148 32460 master_service.cc:960] ReplaceTablet: received request to replace tablet c1faed468bfa4d1ba5d92c0b9b194781 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:54.743986 1600 raft_consensus.cc:2804] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:54.744508 1600 raft_consensus.cc:697] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:54.747076 1600 consensus_queue.cc:237] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:54.758544 792 consensus_peers.cc:597] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:54.761059 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 449359c8e9fa4756ba149f3dce6e0ce7 from {username='slave'} at 127.0.0.1:43806
W20260810 07:52:54.764093 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 285187us
I20260810 07:52:54.767153 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4b3ca46c9c44559761e25bcde3e927" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:54.774863 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet fe43c4aa983843c68a5dcf85201df941 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
W20260810 07:52:54.775581 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet acf740de7abb40bfabfc3a7859cc159b because the tablet was not found. No further retry: Not found: Tablet not found: acf740de7abb40bfabfc3a7859cc159b
W20260810 07:52:54.776768 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 338e9b0e89d44295973d4003abd36649 because the tablet was not found. No further retry: Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
I20260810 07:52:54.784109 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 1 got 0
I20260810 07:52:54.787130 1265 ts_tablet_manager.cc:1929] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:54.790277 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 76) took 2761 ms. Trace:
I20260810 07:52:54.798365 644 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.798646 1047 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:54.799127 1047 ts_tablet_manager.cc:1403] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.111s user 0.012s sys 0.000s
I20260810 07:52:54.799667 644 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.801630 1047 raft_consensus.cc:359] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.806298 1265 log.cc:1199] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/fe43c4aa983843c68a5dcf85201df941
I20260810 07:52:54.808861 1265 ts_tablet_manager.cc:1950] T fe43c4aa983843c68a5dcf85201df941 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:54.816852 1437 rpcz_store.cc:276] 0810 07:52:52.028583 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.028889 (+ 306us) service_pool.cc:224] Handling call
0810 07:52:54.111380 (+2082491us) ts_tablet_manager.cc:1151] Deleting tablet 5b76191c5bed41509344376f1810e7aa
0810 07:52:54.111432 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.445888 (+334456us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.689370 (+243482us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.790201 (+100831us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2081433}
I20260810 07:52:54.813462 1071 tablet_bootstrap.cc:492] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:54.819183 1071 ts_tablet_manager.cc:1403] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.419s user 0.009s sys 0.003s
I20260810 07:52:54.822084 1064 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:54.821754 1071 raft_consensus.cc:359] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.822499 1071 raft_consensus.cc:385] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.822575 1064 ts_tablet_manager.cc:1403] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.175s user 0.011s sys 0.000s
I20260810 07:52:54.822834 1071 raft_consensus.cc:740] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.823611 1071 consensus_queue.cc:260] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.824899 1064 raft_consensus.cc:359] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.791778 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_11b5cd303ac54cd7ac544914b0d3bfac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:54.799683 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09712cfebab34e56ba844e7aa0e8d152 consensus state has no leader
W20260810 07:52:54.830924 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11b5cd303ac54cd7ac544914b0d3bfac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:54.831691 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98 [id=a49a08c7a7954b87aab561894073bf0d]). Expected version 2 got 0
I20260810 07:52:54.832271 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_a448e407c7ec4922ab6bf6b5bb01b6f5 [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 3 got 2
I20260810 07:52:54.835484 1047 raft_consensus.cc:385] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:52:54.836834 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 09712cfebab34e56ba844e7aa0e8d152 (table default.Table_db3c7e150df447dab749a578605e8d4c [id=5fe5c44770974b8eb4f3b55096d25eeb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 09712cfebab34e56ba844e7aa0e8d152 consensus state has no leader
I20260810 07:52:54.826354 639 tablet_service.cc:1511] Processing CreateTablet for tablet b6226a2377864d47b14901df39677127 (DEFAULT_TABLE table=default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:54.837318 1064 raft_consensus.cc:385] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:54.807152 644 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.839128 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6226a2377864d47b14901df39677127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:54.839707 1064 raft_consensus.cc:740] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.852339 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f571ac3561af4ca9b497338a81e1f9a9" } modify_external_catalogs: true
I20260810 07:52:54.853125 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f571ac3561af4ca9b497338a81e1f9a9" } modify_external_catalogs: true
I20260810 07:52:54.853963 1265 tablet_replica.cc:333] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:54.836855 1047 raft_consensus.cc:740] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:54.862984 1047 consensus_queue.cc:260] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:54.848548 573 leader_election.cc:343] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:54.867900 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet fe43c4aa983843c68a5dcf85201df941 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
W20260810 07:52:54.870052 573 consensus_peers.cc:597] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:54.872280 1265 raft_consensus.cc:2243] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.874516 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:54.872635 1047 ts_tablet_manager.cc:1434] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.073s user 0.005s sys 0.000s
I20260810 07:52:54.879459 1047 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:52:54.863384 308 leader_election.cc:343] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:54.883284 308 leader_election.cc:304] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:54.873344 1437 tablet_replica.cc:333] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:54.885239 1265 raft_consensus.cc:2272] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.885511 1437 raft_consensus.cc:2243] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.887728 1265 ts_tablet_manager.cc:1916] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.888670 1437 raft_consensus.cc:2272] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:54.892190 1437 ts_tablet_manager.cc:1916] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:54.893451 1555 raft_consensus.cc:2749] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.861385 1064 consensus_queue.cc:260] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:54.900887 1071 ts_tablet_manager.cc:1434] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.081s user 0.002s sys 0.004s
I20260810 07:52:54.918995 1263 ts_tablet_manager.cc:1929] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:54.951359 1071 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:54.962774 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:52:54.964192 32460 catalog_manager.cc:6807] ReplaceTablet: tablet c1faed468bfa4d1ba5d92c0b9b194781 deleted and replaced by tablet cec929b5f0e14f6586ed50137ecd741b
I20260810 07:52:54.965297 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_14a4faae60bb4b7795842a1172472aeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:54.967686 1263 log.cc:1199] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/69c05d6e49204eedbb305f66dfa2316c
W20260810 07:52:54.968071 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14a4faae60bb4b7795842a1172472aeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:54.968537 1263 ts_tablet_manager.cc:1950] T 69c05d6e49204eedbb305f66dfa2316c P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:54.969348 1064 ts_tablet_manager.cc:1434] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.147s user 0.007s sys 0.000s
I20260810 07:52:54.970307 1064 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:54.970283 644 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:54.975188 384 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:54.979393 1047 tablet_bootstrap.cc:654] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:54.985273 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]) successfully deleted
I20260810 07:52:54.986240 1263 tablet_replica.cc:333] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:52:54.986687 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 because the tablet was not found. No further retry: Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:54.997020 1551 tablet.cc:1724] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:55.000910 868 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:55.002758 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:52:55.004494 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 449359c8e9fa4756ba149f3dce6e0ce7 deleted and replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a
I20260810 07:52:55.008708 384 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:55.011297 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:54.995731 1520 tablet.cc:1724] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:55.006963 644 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:55.016897 1263 raft_consensus.cc:2243] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:54.987452 1437 ts_tablet_manager.cc:1929] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.018275 1437 log.cc:1199] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/1ba026e367ea487fb0dc8aef5d5db606
I20260810 07:52:55.018978 1437 ts_tablet_manager.cc:1950] T 1ba026e367ea487fb0dc8aef5d5db606 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:55.034562 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_94500f9e785c470e8b4ee8981ee5d379" } new_table_name: "default.Table_1af94d6703df4a2eb7e30cdf6301bdce" modify_external_catalogs: true
I20260810 07:52:55.039481 1064 tablet_bootstrap.cc:654] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.042666 1263 raft_consensus.cc:2272] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.047562 32463 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:52:55.050621 1263 ts_tablet_manager.cc:1916] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.052764 384 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:55.052845 868 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:55.054159 644 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:55.054948 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 78) took 2623 ms. Trace:
I20260810 07:52:55.055972 1265 ts_tablet_manager.cc:1929] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:55.064141 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_da3a96ecf800417cbcb7c13b38664a0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:55.064495 1520 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260810 07:52:55.068120 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da3a96ecf800417cbcb7c13b38664a0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.077373 1265 log.cc:1199] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/69c05d6e49204eedbb305f66dfa2316c
I20260810 07:52:55.079416 1265 ts_tablet_manager.cc:1950] T 69c05d6e49204eedbb305f66dfa2316c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:55.080425 1437 rpcz_store.cc:276] 0810 07:52:52.431229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.431362 (+ 133us) service_pool.cc:224] Handling call
0810 07:52:54.818553 (+2387191us) ts_tablet_manager.cc:1151] Deleting tablet 1ba026e367ea487fb0dc8aef5d5db606
0810 07:52:54.818597 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.905437 (+ 86840us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.957285 (+ 51848us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.054884 (+ 97599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2577,"tablet-delete.queue_time_us":2386353}
I20260810 07:52:55.081964 1437 tablet_replica.cc:333] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:55.085654 1437 raft_consensus.cc:2243] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.086292 1437 raft_consensus.cc:2272] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.089157 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 1ba026e367ea487fb0dc8aef5d5db606 (table default.Table_480a23c1ffaa4896a093c62ceff5822e [id=6e2a3445e16944d38471164fb04c71d0]) successfully deleted
I20260810 07:52:55.094504 1639 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:52:55.092187 1643 tablet.cc:1724] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:55.104997 1047 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:55.105602 1047 ts_tablet_manager.cc:1403] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.226s user 0.012s sys 0.003s
I20260810 07:52:55.098536 1551 tablet.cc:1724] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:52:55.099529 1071 tablet_bootstrap.cc:654] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.109921 1552 raft_consensus.cc:493] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.110517 1552 raft_consensus.cc:515] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.113988 1552 leader_election.cc:290] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.114110 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 104) took 1050 ms. Trace:
I20260810 07:52:55.116081 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e2678ab165486dbbb69c7c183a7fa4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:55.116349 574 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
W20260810 07:52:55.121135 573 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.121634 573 leader_election.cc:304] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:55.122416 1552 raft_consensus.cc:2749] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:55.114578 1265 rpcz_store.cc:276] 0810 07:52:54.063778 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.125822 (+ 62044us) service_pool.cc:224] Handling call
0810 07:52:54.853512 (+727690us) ts_tablet_manager.cc:1151] Deleting tablet 69c05d6e49204eedbb305f66dfa2316c
0810 07:52:54.853829 (+ 317us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:54.944342 (+ 90513us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.055757 (+111415us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.114035 (+ 58278us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":726092}
I20260810 07:52:55.124030 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]) successfully deleted
W20260810 07:52:55.126855 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1faed468bfa4d1ba5d92c0b9b194781 already in progress: creating tablet
I20260810 07:52:55.128221 1437 ts_tablet_manager.cc:1916] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.132747 1265 tablet_replica.cc:333] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:55.134096 1265 raft_consensus.cc:2243] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.134579 1265 raft_consensus.cc:2272] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.136309 1555 raft_consensus.cc:493] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.136857 1555 raft_consensus.cc:515] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.137303 1265 ts_tablet_manager.cc:1916] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.139442 1555 leader_election.cc:290] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:55.143375 309 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 4af5109e338844139444bb7111619195
I20260810 07:52:55.145195 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98" } new_table_name: "default.Table_94eb441ee1024892b34ab0ec641b16ff" modify_external_catalogs: true
I20260810 07:52:55.146899 1047 raft_consensus.cc:359] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.147598 1047 raft_consensus.cc:385] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.147861 1047 raft_consensus.cc:740] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.148554 1047 consensus_queue.cc:260] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.152709 1047 ts_tablet_manager.cc:1434] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.047s user 0.006s sys 0.001s
I20260810 07:52:55.155090 1047 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:55.146436 1064 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:55.158587 1064 ts_tablet_manager.cc:1403] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.188s user 0.011s sys 0.004s
I20260810 07:52:55.160422 1071 tablet_bootstrap.cc:492] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:55.156472 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:55.160895 1071 ts_tablet_manager.cc:1403] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.210s user 0.011s sys 0.001s
I20260810 07:52:55.161501 32469 catalog_manager.cc:5719] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:52:55.162303 1064 raft_consensus.cc:359] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.163193 1064 raft_consensus.cc:385] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.163506 1064 raft_consensus.cc:740] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.163298 32469 catalog_manager.cc:5719] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:55.164119 1071 raft_consensus.cc:359] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.164481 1064 consensus_queue.cc:260] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.179827 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af5109e338844139444bb7111619195" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:55.181691 308 leader_election.cc:343] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:52:55.182315 308 leader_election.cc:304] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:55.184211 1555 raft_consensus.cc:2749] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:55.185029 1071 raft_consensus.cc:385] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.185325 1071 raft_consensus.cc:740] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.186007 1071 consensus_queue.cc:260] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.196241 1071 ts_tablet_manager.cc:1434] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260810 07:52:55.197405 1071 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:55.208899 1064 ts_tablet_manager.cc:1434] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.050s user 0.007s sys 0.001s
I20260810 07:52:55.217370 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a8e074b8f7eb43ca90782d7de174dc46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.218922 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8e074b8f7eb43ca90782d7de174dc46 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.226867 1263 ts_tablet_manager.cc:1929] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.227262 1263 log.cc:1199] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/4af5109e338844139444bb7111619195
I20260810 07:52:55.227952 1263 ts_tablet_manager.cc:1950] T 4af5109e338844139444bb7111619195 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:55.239085 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 4af5109e338844139444bb7111619195 (table default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]) successfully deleted
I20260810 07:52:55.216678 1437 ts_tablet_manager.cc:1929] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.245425 32462 master_service.cc:960] ReplaceTablet: received request to replace tablet fe43c4aa983843c68a5dcf85201df941 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:55.245680 1437 log.cc:1199] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/acf740de7abb40bfabfc3a7859cc159b
I20260810 07:52:55.246513 1437 ts_tablet_manager.cc:1950] T acf740de7abb40bfabfc3a7859cc159b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:55.247411 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 79) took 2784 ms. Trace:
I20260810 07:52:55.247702 1437 rpcz_store.cc:276] 0810 07:52:52.462711 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:52.462867 (+ 156us) service_pool.cc:224] Handling call
0810 07:52:55.081794 (+2618927us) ts_tablet_manager.cc:1151] Deleting tablet acf740de7abb40bfabfc3a7859cc159b
0810 07:52:55.081847 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.201550 (+119703us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.216542 (+ 14992us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.247352 (+ 30810us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2617919}
I20260810 07:52:55.249120 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 80) took 1974 ms. Trace:
I20260810 07:52:55.249377 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet acf740de7abb40bfabfc3a7859cc159b (table default.Table_94500f9e785c470e8b4ee8981ee5d379 [id=2fc441ad4da747fc828fd952f1107a0e]) successfully deleted
I20260810 07:52:55.249416 1437 rpcz_store.cc:276] 0810 07:52:53.274547 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.276178 (+ 1631us) service_pool.cc:224] Handling call
0810 07:52:55.248798 (+1972620us) ts_tablet_manager.cc:1151] Deleting tablet 92e2678ab165486dbbb69c7c183a7fa4
0810 07:52:55.248838 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.249069 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1945541}
W20260810 07:52:55.250778 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e2678ab165486dbbb69c7c183a7fa4 already in progress: creating tablet
I20260810 07:52:55.255307 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c82ef7c9085a43d8aa2b06a7799dcab9" } modify_external_catalogs: true
I20260810 07:52:55.255836 1265 ts_tablet_manager.cc:1929] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.255934 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c82ef7c9085a43d8aa2b06a7799dcab9" } modify_external_catalogs: true
I20260810 07:52:55.256217 1265 log.cc:1199] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:52:55.256933 1265 ts_tablet_manager.cc:1950] T 449359c8e9fa4756ba149f3dce6e0ce7 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:55.257684 1064 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:55.260471 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 449359c8e9fa4756ba149f3dce6e0ce7 (table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]) successfully deleted
W20260810 07:52:55.261168 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf45a33485e9463d89b5ce8affa0bd22 already in progress: creating tablet
I20260810 07:52:55.240386 1071 tablet_bootstrap.cc:654] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.243625 1047 tablet_bootstrap.cc:654] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.267472 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 81) took 1990 ms. Trace:
I20260810 07:52:55.253168 1263 tablet_replica.cc:333] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:55.269013 1608 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.269516 1263 raft_consensus.cc:2243] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.269956 1263 raft_consensus.cc:2272] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.269606 1608 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.271704 1263 ts_tablet_manager.cc:1916] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.272269 1608 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.267843 1437 rpcz_store.cc:276] 0810 07:52:53.276859 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.278597 (+ 1738us) service_pool.cc:224] Handling call
0810 07:52:55.266815 (+1988218us) ts_tablet_manager.cc:1151] Deleting tablet f9d3fcf79a104c47a6fd10fc34f7ce08
0810 07:52:55.266872 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.267404 (+ 532us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1952801}
I20260810 07:52:55.274164 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.274672 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:55.275452 676 raft_consensus.cc:2468] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:55.277366 792 leader_election.cc:304] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:52:55.278085 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9d3fcf79a104c47a6fd10fc34f7ce08 already in progress: creating tablet
I20260810 07:52:55.279238 1437 tablet_replica.cc:333] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
W20260810 07:52:55.279145 792 leader_election.cc:343] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.280088 1437 raft_consensus.cc:2243] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.280117 1608 raft_consensus.cc:2804] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.280606 1437 raft_consensus.cc:2272] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.280647 1608 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.281262 1608 raft_consensus.cc:3060] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:55.282404 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet afbac73bbeb44af6b39ab23c5cf79eed consensus state has no leader
I20260810 07:52:55.289840 1437 ts_tablet_manager.cc:1916] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.302946 1047 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:55.303391 1047 ts_tablet_manager.cc:1403] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.148s user 0.006s sys 0.005s
I20260810 07:52:55.306696 1047 raft_consensus.cc:359] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.307318 1047 raft_consensus.cc:385] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.307569 1047 raft_consensus.cc:740] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.320089 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_fb6ddd83c8f44109b5a1be3203096778"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.321812 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb6ddd83c8f44109b5a1be3203096778 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.308184 1047 consensus_queue.cc:260] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.332949 1047 ts_tablet_manager.cc:1434] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.029s user 0.006s sys 0.002s
I20260810 07:52:55.343971 1552 raft_consensus.cc:493] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.344645 1552 raft_consensus.cc:515] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.347823 1552 leader_election.cc:290] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.348404 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.349423 1047 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:52:55.350584 573 leader_election.cc:343] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.353425 1071 tablet_bootstrap.cc:492] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:55.353896 1071 ts_tablet_manager.cc:1403] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.157s user 0.023s sys 0.006s
I20260810 07:52:55.358315 1071 raft_consensus.cc:359] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.360052 1071 raft_consensus.cc:385] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.360381 1071 raft_consensus.cc:740] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.361083 1071 consensus_queue.cc:260] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.369735 1654 raft_consensus.cc:493] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.370329 1263 ts_tablet_manager.cc:1929] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.370823 1263 log.cc:1199] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/4ec00a494816409684205c7f4435a079
I20260810 07:52:55.370409 1654 raft_consensus.cc:515] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.371564 1263 ts_tablet_manager.cc:1950] T 4ec00a494816409684205c7f4435a079 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:55.374910 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:55.375825 892 raft_consensus.cc:2468] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:55.377226 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.379561 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 4ec00a494816409684205c7f4435a079 (table default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]) successfully deleted
I20260810 07:52:55.380498 574 leader_election.cc:304] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
W20260810 07:52:55.381373 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1faed468bfa4d1ba5d92c0b9b194781 already in progress: creating tablet
I20260810 07:52:55.381793 1552 raft_consensus.cc:2804] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.382228 1552 raft_consensus.cc:493] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.382615 1552 raft_consensus.cc:3060] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:55.382838 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 449359c8e9fa4756ba149f3dce6e0ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449359c8e9fa4756ba149f3dce6e0ce7 already in progress: creating tablet
I20260810 07:52:55.384070 1437 ts_tablet_manager.cc:1929] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.384532 1437 log.cc:1199] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/a1373ebbd3ba4262806bfc17823d25b5
W20260810 07:52:55.385851 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf45a33485e9463d89b5ce8affa0bd22 already in progress: creating tablet
I20260810 07:52:55.388244 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b9d12e0451244cc2967c89a27688e75d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.391486 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9d12e0451244cc2967c89a27688e75d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:55.393496 792 leader_election.cc:343] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.379372 1655 raft_consensus.cc:493] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.395442 1047 tablet_bootstrap.cc:654] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.396667 1552 raft_consensus.cc:515] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.400738 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:55.395141 1655 raft_consensus.cc:515] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:55.403131 573 leader_election.cc:343] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.411011 1064 tablet_bootstrap.cc:654] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.414414 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:55.415167 892 raft_consensus.cc:3060] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.424942 1608 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.427479 1608 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.429199 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:55.429368 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:55.430053 676 raft_consensus.cc:3060] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.439107 676 raft_consensus.cc:2468] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:55.441994 1437 ts_tablet_manager.cc:1950] T a1373ebbd3ba4262806bfc17823d25b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:55.443022 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 87) took 1513 ms. Trace:
I20260810 07:52:55.443338 1437 rpcz_store.cc:276] 0810 07:52:53.929689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.929825 (+ 136us) service_pool.cc:224] Handling call
0810 07:52:55.279061 (+1349236us) ts_tablet_manager.cc:1151] Deleting tablet a1373ebbd3ba4262806bfc17823d25b5
0810 07:52:55.279110 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.330444 (+ 51334us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.383889 (+ 53445us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.442959 (+ 59070us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1348657}
I20260810 07:52:55.444599 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet a1373ebbd3ba4262806bfc17823d25b5 (table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]) successfully deleted
I20260810 07:52:55.456244 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 86) took 1526 ms. Trace:
I20260810 07:52:55.456558 1437 rpcz_store.cc:276] 0810 07:52:53.929395 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.930166 (+ 771us) service_pool.cc:224] Handling call
0810 07:52:55.455979 (+1525813us) ts_tablet_manager.cc:1151] Deleting tablet f353e1e07485400a964f8e05b0691cc6
0810 07:52:55.456022 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.456184 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1525222}
W20260810 07:52:55.457665 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet f353e1e07485400a964f8e05b0691cc6 because the tablet was not found. No further retry: Not found: Tablet not found: f353e1e07485400a964f8e05b0691cc6
I20260810 07:52:55.459057 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 88) took 1529 ms. Trace:
I20260810 07:52:55.459515 1552 leader_election.cc:290] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.459365 1437 rpcz_store.cc:276] 0810 07:52:53.930031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:53.930894 (+ 863us) service_pool.cc:224] Handling call
0810 07:52:55.458730 (+1527836us) ts_tablet_manager.cc:1151] Deleting tablet 338e9b0e89d44295973d4003abd36649
0810 07:52:55.458768 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.459005 (+ 237us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527349}
I20260810 07:52:55.456041 792 leader_election.cc:304] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:52:55.460598 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 338e9b0e89d44295973d4003abd36649 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 338e9b0e89d44295973d4003abd36649 already in progress: creating tablet
I20260810 07:52:55.468451 1437 tablet_replica.cc:333] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
W20260810 07:52:55.469735 792 leader_election.cc:343] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.459210 1071 ts_tablet_manager.cc:1434] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.105s user 0.007s sys 0.000s
I20260810 07:52:55.472289 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 5195424dd98f444caa75e2ca98d09e6d from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:55.490439 1552 raft_consensus.cc:493] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.491022 1047 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:55.478211 1437 raft_consensus.cc:2243] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.491629 1437 raft_consensus.cc:2272] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.492174 32460 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_7423fb2519b540d6a3f8364376cd7148" } new_table_name: "default.Table_956f0c9ce19f4e929b17f81669a8e797" modify_external_catalogs: true
I20260810 07:52:55.491465 1047 ts_tablet_manager.cc:1403] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.142s user 0.005s sys 0.007s
I20260810 07:52:55.495096 1437 ts_tablet_manager.cc:1916] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.496532 1047 raft_consensus.cc:359] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.500506 1047 raft_consensus.cc:385] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.500808 1047 raft_consensus.cc:740] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.500977 32463 master_service.cc:960] ReplaceTablet: received request to replace tablet 338e9b0e89d44295973d4003abd36649 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:55.501434 1047 consensus_queue.cc:260] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.509804 1071 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:55.491196 1552 raft_consensus.cc:515] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.513796 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:55.516654 573 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.519670 1552 leader_election.cc:290] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.527205 1047 ts_tablet_manager.cc:1434] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260810 07:52:55.527591 1656 raft_consensus.cc:2804] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:55.529316 1655 leader_election.cc:290] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.531340 1047 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:55.534750 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 5195424dd98f444caa75e2ca98d09e6d
I20260810 07:52:55.536675 384 tablet_service.cc:1558] Processing DeleteTablet for tablet 5195424dd98f444caa75e2ca98d09e6d with delete_type TABLET_DATA_DELETED (replaced by tablet 62352629fce54deebaa507f58ff908e6) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:55.534543 1654 leader_election.cc:290] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.521340 892 raft_consensus.cc:2468] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:55.546032 1656 raft_consensus.cc:697] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:55.545920 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_af59b24731ec4cf8a7c32525e4891ee7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:55.545331 1437 ts_tablet_manager.cc:1929] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:55.548113 1047 tablet_bootstrap.cc:654] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.547283 1656 consensus_queue.cc:237] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.548520 1437 log.cc:1199] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/fe43c4aa983843c68a5dcf85201df941
I20260810 07:52:55.550364 1437 ts_tablet_manager.cc:1950] T fe43c4aa983843c68a5dcf85201df941 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:52:55.554129 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 62136.3us
I20260810 07:52:55.555768 574 leader_election.cc:304] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.556214 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.556332 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:55.557044 678 raft_consensus.cc:2468] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:55.557377 1600 raft_consensus.cc:2804] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:55.557998 1600 raft_consensus.cc:697] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:55.559031 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:55.559203 1600 consensus_queue.cc:237] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.559710 892 raft_consensus.cc:2468] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:55.560484 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:55.561705 792 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.562337 792 leader_election.cc:304] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
W20260810 07:52:55.560640 573 consensus_peers.cc:597] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:55.560937 574 leader_election.cc:304] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.564553 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 92) took 1476 ms. Trace:
I20260810 07:52:55.565433 676 raft_consensus.cc:2468] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
W20260810 07:52:55.574858 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af59b24731ec4cf8a7c32525e4891ee7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.566051 1437 rpcz_store.cc:276] 0810 07:52:54.087860 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.092831 (+ 4971us) service_pool.cc:224] Handling call
0810 07:52:55.468275 (+1375444us) ts_tablet_manager.cc:1151] Deleting tablet fe43c4aa983843c68a5dcf85201df941
0810 07:52:55.468325 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.523872 (+ 55547us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.545193 (+ 21321us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.564414 (+ 19221us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1366043}
I20260810 07:52:55.589026 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet fe43c4aa983843c68a5dcf85201df941 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
I20260810 07:52:55.590301 1437 tablet_replica.cc:333] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:55.592058 1437 raft_consensus.cc:2243] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.593043 1437 raft_consensus.cc:2272] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.593564 1064 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:55.593997 1064 ts_tablet_manager.cc:1403] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.337s user 0.014s sys 0.002s
I20260810 07:52:55.597656 1667 raft_consensus.cc:2804] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.598412 1667 raft_consensus.cc:493] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.598802 1667 raft_consensus.cc:3060] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.605724 1437 ts_tablet_manager.cc:1916] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.625590 1655 raft_consensus.cc:2804] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.626003 1655 raft_consensus.cc:493] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.626371 1655 raft_consensus.cc:3060] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.629325 1662 raft_consensus.cc:493] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.629854 1662 raft_consensus.cc:515] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.630609 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 5195424dd98f444caa75e2ca98d09e6d deleted and replaced by tablet 62352629fce54deebaa507f58ff908e6
I20260810 07:52:55.634359 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 5195424dd98f444caa75e2ca98d09e6d with delete_type TABLET_DATA_DELETED (replaced by tablet 62352629fce54deebaa507f58ff908e6) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:55.636857 1263 tablet_replica.cc:333] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:52:55.637130 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 74880.1us
I20260810 07:52:55.637590 1263 raft_consensus.cc:2243] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:55.637879 792 leader_election.cc:304] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.638415 1263 raft_consensus.cc:2272] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.638903 1656 raft_consensus.cc:2804] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.639364 1656 raft_consensus.cc:493] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.639796 1656 raft_consensus.cc:3060] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.641924 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 5195424dd98f444caa75e2ca98d09e6d with delete_type TABLET_DATA_DELETED (replaced by tablet 62352629fce54deebaa507f58ff908e6) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:55.640496 1263 ts_tablet_manager.cc:1916] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.642657 1265 tablet_replica.cc:333] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:55.643388 1265 raft_consensus.cc:2243] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.643888 1265 raft_consensus.cc:2272] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.646075 1265 ts_tablet_manager.cc:1916] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:55.650163 1667 raft_consensus.cc:515] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.653213 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:55.653263 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:55.654129 1662 leader_election.cc:290] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:55.655038 573 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.655491 1064 raft_consensus.cc:359] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.655890 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.656114 1064 raft_consensus.cc:385] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.656365 1064 raft_consensus.cc:740] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.656953 1064 consensus_queue.cc:260] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.658687 1047 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:55.660171 1047 ts_tablet_manager.cc:1403] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.129s user 0.004s sys 0.014s
I20260810 07:52:55.666731 1047 raft_consensus.cc:359] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.667445 1047 raft_consensus.cc:385] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.667718 1047 raft_consensus.cc:740] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.668417 1047 consensus_queue.cc:260] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.669292 1667 leader_election.cc:290] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.671818 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_8bed0806196e4e6393b129851ab29404"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:55.672930 1265 ts_tablet_manager.cc:1929] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:55.673300 1265 log.cc:1199] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/5195424dd98f444caa75e2ca98d09e6d
I20260810 07:52:55.673460 1071 tablet_bootstrap.cc:654] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:55.673496 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8bed0806196e4e6393b129851ab29404 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.674006 1265 ts_tablet_manager.cc:1950] T 5195424dd98f444caa75e2ca98d09e6d P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:55.678655 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_34a402e90a4048849f6098cd5893fe31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:55.679592 1655 raft_consensus.cc:515] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.680691 892 raft_consensus.cc:2393] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:52:55.683916 1047 ts_tablet_manager.cc:1434] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260810 07:52:55.684830 1047 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:55.689834 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:55.697839 574 leader_election.cc:304] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:55.697853 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_b0bcb2fbc56b4af8a03647a3c1e36d98 [id=a49a08c7a7954b87aab561894073bf0d]) successfully deleted
W20260810 07:52:55.698020 792 leader_election.cc:343] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:55.699195 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34a402e90a4048849f6098cd5893fe31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:55.700337 792 leader_election.cc:343] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.700433 1671 raft_consensus.cc:2749] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:55.700693 792 leader_election.cc:304] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:55.701458 1662 raft_consensus.cc:2749] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:55.726971 1655 leader_election.cc:290] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.729740 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:55.730175 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09712cfebab34e56ba844e7aa0e8d152" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:55.731117 676 raft_consensus.cc:2393] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:55.735232 1656 raft_consensus.cc:515] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.737317 1656 leader_election.cc:290] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:55.739048 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:55.739125 1667 raft_consensus.cc:493] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.739652 676 raft_consensus.cc:3060] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.741266 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:55.739897 1667 raft_consensus.cc:515] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.744755 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.745350 1667 leader_election.cc:290] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.746168 1047 tablet_bootstrap.cc:654] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:55.747484 792 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:55.746177 573 leader_election.cc:343] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.747887 792 leader_election.cc:304] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:52:55.748960 792 leader_election.cc:343] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.746711 676 raft_consensus.cc:2468] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:55.757117 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:55.757743 892 raft_consensus.cc:2468] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:55.758882 574 leader_election.cc:304] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.760228 792 leader_election.cc:304] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.760921 1667 raft_consensus.cc:2804] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:55.761348 1667 raft_consensus.cc:493] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:55.761767 1667 raft_consensus.cc:3060] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.767620 1655 raft_consensus.cc:2749] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:52:55.768215 1655 raft_consensus.cc:2804] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:55.771622 1667 raft_consensus.cc:515] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.772152 1655 raft_consensus.cc:697] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:55.773000 1655 consensus_queue.cc:237] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.774554 1047 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:55.774992 1047 ts_tablet_manager.cc:1403] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.090s user 0.011s sys 0.002s
I20260810 07:52:55.777251 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:55.777349 1047 raft_consensus.cc:359] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.777814 892 raft_consensus.cc:3060] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:55.777948 1047 raft_consensus.cc:385] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:55.778190 1047 raft_consensus.cc:740] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:55.778764 1047 consensus_queue.cc:260] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.779074 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:55.781315 573 leader_election.cc:343] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.774575 1667 leader_election.cc:290] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.782274 1047 ts_tablet_manager.cc:1434] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:52:55.785974 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0b54380ef2094266b129d78070f1fcab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:55.787390 1555 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:55.787850 1555 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.789810 1555 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:55.792912 384 tablet_service.cc:1511] Processing CreateTablet for tablet 37b48c88571040f6be8d7cbd388feb58 (DEFAULT_TABLE table=default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.794072 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b48c88571040f6be8d7cbd388feb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.794799 892 raft_consensus.cc:2468] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:55.803721 639 tablet_service.cc:1511] Processing CreateTablet for tablet 37b48c88571040f6be8d7cbd388feb58 (DEFAULT_TABLE table=default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.804697 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b48c88571040f6be8d7cbd388feb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.805192 1047 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:55.802389 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:55.805907 896 raft_consensus.cc:2393] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:52:55.807050 1064 ts_tablet_manager.cc:1434] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.213s user 0.007s sys 0.000s
I20260810 07:52:55.807987 1064 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:55.811812 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b54380ef2094266b129d78070f1fcab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.813419 574 leader_election.cc:304] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:55.814670 1667 raft_consensus.cc:2804] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:55.815092 1667 raft_consensus.cc:697] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:55.815843 1667 consensus_queue.cc:237] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:55.824608 385 tablet_service.cc:1511] Processing CreateTablet for tablet ab2e7619a25b437c8ee2c817255ca638 (DEFAULT_TABLE table=default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.825734 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2e7619a25b437c8ee2c817255ca638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.831346 1047 tablet_bootstrap.cc:654] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.834585 868 tablet_service.cc:1511] Processing CreateTablet for tablet 37b48c88571040f6be8d7cbd388feb58 (DEFAULT_TABLE table=default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.835615 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b48c88571040f6be8d7cbd388feb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.845638 872 tablet_service.cc:1511] Processing CreateTablet for tablet ab2e7619a25b437c8ee2c817255ca638 (DEFAULT_TABLE table=default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.846632 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2e7619a25b437c8ee2c817255ca638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.851231 375 tablet_service.cc:1511] Processing CreateTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 (DEFAULT_TABLE table=default.Table_58600ebaa02642eea48fbf663c45afe2 [id=ad1d545c245b45d19afe4cd8816fa3fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.852392 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04e29d3a49b4bd891be79e0fad6ec51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.855257 32464 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 1 got 0
I20260810 07:52:55.856215 32464 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]). Expected version 3 got 2
I20260810 07:52:55.857417 32464 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_7423fb2519b540d6a3f8364376cd7148 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 1 got 0
I20260810 07:52:55.857734 382 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe1a6581374213be7c3f63a7a567a4 (DEFAULT_TABLE table=default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.858333 32464 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 09712cfebab34e56ba844e7aa0e8d152 (table default.Table_db3c7e150df447dab749a578605e8d4c [id=5fe5c44770974b8eb4f3b55096d25eeb]). Expected version 1 got 0
I20260810 07:52:55.860409 860 tablet_service.cc:1511] Processing CreateTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 (DEFAULT_TABLE table=default.Table_58600ebaa02642eea48fbf663c45afe2 [id=ad1d545c245b45d19afe4cd8816fa3fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.861531 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04e29d3a49b4bd891be79e0fad6ec51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.864574 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe1a6581374213be7c3f63a7a567a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.867664 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:52:55.870604 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:55.871268 871 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:55.871996 1265 tablet_replica.cc:333] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:55.872738 1265 raft_consensus.cc:2243] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:55.873171 1265 raft_consensus.cc:2272] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:55.875442 1265 ts_tablet_manager.cc:1916] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:55.908216 32461 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62352629fce54deebaa507f58ff908e6 has no consensus state
I20260810 07:52:55.911764 32467 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:52:55.911984 1437 ts_tablet_manager.cc:1929] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:52:55.908138 872 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe1a6581374213be7c3f63a7a567a4 (DEFAULT_TABLE table=default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.912384 1437 log.cc:1199] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/69c05d6e49204eedbb305f66dfa2316c
I20260810 07:52:55.913525 32467 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 149) took 1084 ms. Trace:
I20260810 07:52:55.915277 1071 tablet_bootstrap.cc:492] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:55.915784 1071 ts_tablet_manager.cc:1403] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.406s user 0.008s sys 0.005s
I20260810 07:52:55.915459 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe1a6581374213be7c3f63a7a567a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.918365 1064 tablet_bootstrap.cc:654] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:55.913821 32467 rpcz_store.cc:276] 0810 07:52:54.829609 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.831139 (+ 1530us) service_pool.cc:224] Handling call
0810 07:52:54.852232 (+ 21093us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c4d6 00007f30cf424b34 00007f30cf424ac2 00007f30cf4249f1 00007f30cf42496c 00007f30cf423c22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:54.853634 (+ 1402us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:54.854194 (+ 560us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:52:54.854326 (+ 132us) catalog_manager.cc:2850] Locking tablets
0810 07:52:54.948221 (+ 93895us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:52:55.152134 (+203913us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:52:55.152390 (+ 256us) catalog_manager.cc:2898] Committing in-memory state
0810 07:52:55.911661 (+759271us) catalog_manager.cc:2902] Aborting table tasks
0810 07:52:55.913466 (+ 1805us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:55.108633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:55.109918 (+ 1285us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:55.109941 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:52:55.109959 (+ 18us) tablet.cc:662] Decoding operations
0810 07:52:55.110376 (+ 417us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:55.110417 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:55.110437 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:55.110845 (+ 408us) tablet.cc:701] Row locks acquired
0810 07:52:55.110870 (+ 25us) write_op.cc:260] PREPARE: finished
0810 07:52:55.110992 (+ 122us) write_op.cc:270] Start()
0810 07:52:55.111134 (+ 142us) write_op.cc:276] Timestamp: P: 1786348375110940 usec, L: 0
0810 07:52:55.111151 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:52:55.111821 (+ 670us) log.cc:844] Serialized 731 byte log entry
0810 07:52:55.149080 (+ 37259us) op_driver.cc:464] REPLICATION: finished
0810 07:52:55.150427 (+ 1347us) write_op.cc:301] APPLY: starting
0810 07:52:55.150491 (+ 64us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:55.150609 (+ 118us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:55.150715 (+ 106us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:55.150984 (+ 269us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:55.151142 (+ 158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:55.151189 (+ 47us) write_op.cc:312] APPLY: finished
0810 07:52:55.151450 (+ 261us) log.cc:844] Serialized 29 byte log entry
0810 07:52:55.151689 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:55.151787 (+ 98us) write_op.cc:454] Released schema lock
0810 07:52:55.151894 (+ 107us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:55.152234 (+ 340us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:55.152258 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":46631296,"child_traces":[["op",{"apply.queue_time_us":1185,"apply.run_cpu_time_us":1737,"apply.run_wall_time_us":1737,"num_ops":2,"prepare.queue_time_us":157969,"prepare.run_cpu_time_us":2696,"prepare.run_wall_time_us":13631,"replication_time_us":37828}]]}
I20260810 07:52:55.924865 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_088fa7381e654b13b33b9eac210aec71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.926515 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_088fa7381e654b13b33b9eac210aec71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:55.932168 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 62352629fce54deebaa507f58ff908e6 (table default.Table_94eb441ee1024892b34ab0ec641b16ff [id=a49a08c7a7954b87aab561894073bf0d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62352629fce54deebaa507f58ff908e6 has no consensus state
I20260810 07:52:55.943301 32463 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 1 got 0
I20260810 07:52:55.944049 32465 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_a448e407c7ec4922ab6bf6b5bb01b6f5 [id=6ed43022a4d74f538afc9f9786e629e1] alter complete (version 3)
W20260810 07:52:55.944628 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 32) took 1234 ms (client timeout 1000 ms). Trace:
W20260810 07:52:55.944962 32465 rpcz_store.cc:269] 0810 07:52:54.710551 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.738838 (+ 28287us) service_pool.cc:224] Handling call
0810 07:52:54.800360 (+ 61522us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:55.944562 (+1144202us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:55.038543 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:55.066392 (+ 27849us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:55.066419 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:52:55.066455 (+ 36us) tablet.cc:662] Decoding operations
0810 07:52:55.066728 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:55.066777 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:55.066810 (+ 33us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:55.066980 (+ 170us) tablet.cc:701] Row locks acquired
0810 07:52:55.066993 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:55.067140 (+ 147us) write_op.cc:270] Start()
0810 07:52:55.067221 (+ 81us) write_op.cc:276] Timestamp: P: 1786348375067093 usec, L: 0
0810 07:52:55.067238 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:52:55.067717 (+ 479us) log.cc:844] Serialized 356 byte log entry
0810 07:52:55.093787 (+ 26070us) op_driver.cc:464] REPLICATION: finished
0810 07:52:55.094134 (+ 347us) write_op.cc:301] APPLY: starting
0810 07:52:55.094189 (+ 55us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:55.094304 (+ 115us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:55.094601 (+ 297us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:55.094897 (+ 296us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:55.095155 (+ 258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:55.095244 (+ 89us) write_op.cc:312] APPLY: finished
0810 07:52:55.095548 (+ 304us) log.cc:844] Serialized 23 byte log entry
0810 07:52:55.095880 (+ 332us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:55.096009 (+ 129us) write_op.cc:454] Released schema lock
0810 07:52:55.096061 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:55.108337 (+ 12276us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:55.108361 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":85601664,"child_traces":[["op",{"apply.queue_time_us":180,"apply.run_cpu_time_us":2260,"apply.run_wall_time_us":2377,"num_ops":1,"prepare.queue_time_us":89742,"prepare.run_cpu_time_us":2017,"prepare.run_wall_time_us":69761,"replication_time_us":26455}]]}
I20260810 07:52:55.951326 380 tablet_service.cc:1511] Processing CreateTablet for tablet 54a3bd8369634e4bbfdf8920454357bc (DEFAULT_TABLE table=default.Table_0f57e2f93400402d8c7458cf70c65ccc [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.952564 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a3bd8369634e4bbfdf8920454357bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.964481 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_371cedf390b0495f84d3da22c3b205ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.966006 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_371cedf390b0495f84d3da22c3b205ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:55.983425 385 tablet_service.cc:1511] Processing CreateTablet for tablet dd22d1029ff249a0a9e05862293261ff (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.984273 32461 master_service.cc:960] ReplaceTablet: received request to replace tablet b2bfffdd01524eada0a85c1a525325a0 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:55.986003 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd22d1029ff249a0a9e05862293261ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:55.990330 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e44a2c3484b947aa9c4953108b45c259"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:55.992170 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e44a2c3484b947aa9c4953108b45c259 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:56.006716 32450 catalog_manager.cc:5123] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): alter failed for tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_94eb441ee1024892b34ab0ec641b16ff [id=a49a08c7a7954b87aab561894073bf0d]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:52:55.833959 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:56.016583 676 raft_consensus.cc:2393] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:55.840459 638 tablet_service.cc:1511] Processing CreateTablet for tablet ab2e7619a25b437c8ee2c817255ca638 (DEFAULT_TABLE table=default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.852916 645 tablet_service.cc:1511] Processing CreateTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 (DEFAULT_TABLE table=default.Table_58600ebaa02642eea48fbf663c45afe2 [id=ad1d545c245b45d19afe4cd8816fa3fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:55.869850 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:55.891120 1671 raft_consensus.cc:493] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.018244 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2e7619a25b437c8ee2c817255ca638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.018091 1671 raft_consensus.cc:515] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.020289 1671 leader_election.cc:290] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:56.021286 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a04e29d3a49b4bd891be79e0fad6ec51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.023324 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:55.907604 644 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe1a6581374213be7c3f63a7a567a4 (DEFAULT_TABLE table=default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.024838 308 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:56.025969 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe1a6581374213be7c3f63a7a567a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.026336 1555 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:56.027552 573 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:55.918334 1071 raft_consensus.cc:359] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:56.034899 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 22) took 1163 ms (client timeout 1000 ms). Trace:
I20260810 07:52:55.913053 1667 consensus_queue.cc:1048] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:55.927829 635 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:55.957545 642 tablet_service.cc:1511] Processing CreateTablet for tablet 54a3bd8369634e4bbfdf8920454357bc (DEFAULT_TABLE table=default.Table_0f57e2f93400402d8c7458cf70c65ccc [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.037694 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a3bd8369634e4bbfdf8920454357bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.037825 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57706
W20260810 07:52:56.035247 32469 rpcz_store.cc:269] 0810 07:52:54.871598 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.871703 (+ 105us) service_pool.cc:224] Handling call
0810 07:52:54.953443 (+ 81740us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:56.034823 (+1081380us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:55.381350 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:55.381738 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:55.381760 (+ 22us) write_op.cc:435] Acquired schema lock
0810 07:52:55.381773 (+ 13us) tablet.cc:662] Decoding operations
0810 07:52:55.382073 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:55.382126 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:55.382148 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:55.382548 (+ 400us) tablet.cc:701] Row locks acquired
0810 07:52:55.382591 (+ 43us) write_op.cc:260] PREPARE: finished
0810 07:52:55.382816 (+ 225us) write_op.cc:270] Start()
0810 07:52:55.385565 (+ 2749us) write_op.cc:276] Timestamp: P: 1786348375382710 usec, L: 0
0810 07:52:55.385584 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:52:55.386106 (+ 522us) log.cc:844] Serialized 913 byte log entry
0810 07:52:55.439555 (+ 53449us) op_driver.cc:464] REPLICATION: finished
0810 07:52:55.439789 (+ 234us) write_op.cc:301] APPLY: starting
0810 07:52:55.439852 (+ 63us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:55.440005 (+ 153us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:55.440120 (+ 115us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:55.440399 (+ 279us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:55.440580 (+ 181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:55.440645 (+ 65us) write_op.cc:312] APPLY: finished
0810 07:52:55.440905 (+ 260us) log.cc:844] Serialized 29 byte log entry
0810 07:52:55.441069 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:55.441142 (+ 73us) write_op.cc:454] Released schema lock
0810 07:52:55.441177 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":79946,"reported_tablets":10,"spinlock_wait_cycles":181918080,"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":1625,"apply.run_wall_time_us":1625,"num_ops":2,"prepare.queue_time_us":180528,"prepare.run_cpu_time_us":2442,"prepare.run_wall_time_us":7478,"raft.queue_time_us":91,"raft.run_cpu_time_us":1246,"raft.run_wall_time_us":1246,"replication_time_us":53882}]]}
I20260810 07:52:56.045665 635 tablet_service.cc:1511] Processing CreateTablet for tablet dd22d1029ff249a0a9e05862293261ff (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.046986 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd22d1029ff249a0a9e05862293261ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.066767 32460 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_680195385fd2411baf85a6a49bab6feb" } new_table_name: "default.Table_5f7e619b24fe4837930915dbe61bd2b8" modify_external_catalogs: true
I20260810 07:52:56.074651 377 tablet_service.cc:1511] Processing CreateTablet for tablet f8415b6376a740388487f7ae4aebea52 (DEFAULT_TABLE table=default.Table_b041641ad99146e8ac57b26f842b8976 [id=0ab295f37b5e482482053f67fe0719a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.075776 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8415b6376a740388487f7ae4aebea52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.076924 1686 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:56.072604 1071 raft_consensus.cc:385] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.041368 384 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:56.034726 1555 raft_consensus.cc:2749] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:56.095232 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 19) took 2113 ms (client timeout 1000 ms). Trace:
W20260810 07:52:56.095687 32470 rpcz_store.cc:269] 0810 07:52:53.981674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.037301 (+ 55627us) service_pool.cc:224] Handling call
0810 07:52:56.095154 (+2057853us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64599,"reported_tablets":17}
I20260810 07:52:56.098145 1071 raft_consensus.cc:740] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:56.098496 32470 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_94eb441ee1024892b34ab0ec641b16ff" } modify_external_catalogs: true
I20260810 07:52:56.099002 32470 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_94eb441ee1024892b34ab0ec641b16ff" } modify_external_catalogs: true
I20260810 07:52:56.100132 1071 consensus_queue.cc:260] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.108448 1263 ts_tablet_manager.cc:1929] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:56.108865 1263 log.cc:1199] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/5195424dd98f444caa75e2ca98d09e6d
I20260810 07:52:56.109766 1263 ts_tablet_manager.cc:1950] T 5195424dd98f444caa75e2ca98d09e6d P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:55.927052 860 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:55.929572 1265 ts_tablet_manager.cc:1929] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:56.111322 1265 log.cc:1199] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:52:56.112059 1265 ts_tablet_manager.cc:1950] T 2c171d3358e44ae2962ab3c4c377b0f1 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:56.065287 1688 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:56.114305 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 09712cfebab34e56ba844e7aa0e8d152 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:56.114792 1265 tablet_replica.cc:333] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:55.973863 872 tablet_service.cc:1511] Processing CreateTablet for tablet 54a3bd8369634e4bbfdf8920454357bc (DEFAULT_TABLE table=default.Table_0f57e2f93400402d8c7458cf70c65ccc [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.008416 871 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.021008 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ba026e367ea487fb0dc8aef5d5db606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.021320 866 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.021548 862 tablet_service.cc:1558] Processing DeleteTablet for tablet 69c05d6e49204eedbb305f66dfa2316c with delete_type TABLET_DATA_DELETED (replaced by tablet 8718f998623e4f1c8b7b8be815e184a3) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.022787 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:56.115947 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a3bd8369634e4bbfdf8920454357bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.116083 896 raft_consensus.cc:2468] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:56.077556 869 tablet_service.cc:1511] Processing CreateTablet for tablet dd22d1029ff249a0a9e05862293261ff (DEFAULT_TABLE table=default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.117332 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd22d1029ff249a0a9e05862293261ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.117247 574 leader_election.cc:304] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:56.118278 1667 raft_consensus.cc:2804] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:56.118626 1667 raft_consensus.cc:493] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:56.118954 1667 raft_consensus.cc:3060] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:56.121743 1265 raft_consensus.cc:2243] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:56.122399 1265 raft_consensus.cc:2272] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:56.126662 1667 raft_consensus.cc:515] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.129832 1667 leader_election.cc:290] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:56.129884 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:56.129993 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfffdd01524eada0a85c1a525325a0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:56.130568 896 raft_consensus.cc:3060] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:56.130975 1265 ts_tablet_manager.cc:1916] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:56.131570 573 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:56.131843 1071 ts_tablet_manager.cc:1434] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.216s user 0.006s sys 0.000s
I20260810 07:52:56.133546 1071 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:56.138033 896 raft_consensus.cc:2468] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:56.139573 574 leader_election.cc:304] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:56.147722 1667 raft_consensus.cc:2804] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:56.151753 1667 raft_consensus.cc:697] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:56.152855 1667 consensus_queue.cc:237] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.163837 1265 ts_tablet_manager.cc:1929] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:56.164263 1265 log.cc:1199] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:52:56.165060 1265 ts_tablet_manager.cc:1950] T c7c136422ffe4bb2b4f77ab081acd19f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:56.180557 32461 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet b2bfffdd01524eada0a85c1a525325a0
I20260810 07:52:56.184309 32461 catalog_manager.cc:6807] ReplaceTablet: tablet b2bfffdd01524eada0a85c1a525325a0 deleted and replaced by tablet 4c13f622081847f88082489de32d58df
I20260810 07:52:56.187381 1437 ts_tablet_manager.cc:1950] T 69c05d6e49204eedbb305f66dfa2316c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:56.193660 872 tablet_service.cc:1511] Processing CreateTablet for tablet f8415b6376a740388487f7ae4aebea52 (DEFAULT_TABLE table=default.Table_b041641ad99146e8ac57b26f842b8976 [id=0ab295f37b5e482482053f67fe0719a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.194729 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8415b6376a740388487f7ae4aebea52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.194980 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 94) took 2106 ms. Trace:
I20260810 07:52:56.195287 1437 rpcz_store.cc:276] 0810 07:52:54.088466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.101863 (+ 13397us) service_pool.cc:224] Handling call
0810 07:52:55.590019 (+1488156us) ts_tablet_manager.cc:1151] Deleting tablet 69c05d6e49204eedbb305f66dfa2316c
0810 07:52:55.590097 (+ 78us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:55.665549 (+ 75452us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:55.911828 (+246279us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:56.194917 (+283089us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1487551}
I20260810 07:52:56.101573 868 tablet_service.cc:1558] Processing DeleteTablet for tablet acf740de7abb40bfabfc3a7859cc159b with delete_type TABLET_DATA_DELETED (replaced by tablet afbac73bbeb44af6b39ab23c5cf79eed) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.201126 1071 tablet_bootstrap.cc:654] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:56.206815 868 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.221112 1265 tablet_replica.cc:333] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:56.221909 1265 raft_consensus.cc:2243] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:56.222343 1265 raft_consensus.cc:2272] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:56.239248 1071 tablet_bootstrap.cc:492] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:56.239686 1071 ts_tablet_manager.cc:1403] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.107s user 0.010s sys 0.002s
I20260810 07:52:56.243642 1263 tablet_replica.cc:333] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:56.254971 1071 raft_consensus.cc:359] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.255635 1071 raft_consensus.cc:385] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.255893 1071 raft_consensus.cc:740] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:56.257902 1071 consensus_queue.cc:260] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.262519 1265 ts_tablet_manager.cc:1916] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:56.267973 1688 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:56.277684 1437 tablet_replica.cc:333] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:56.280001 1071 ts_tablet_manager.cc:1434] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.040s user 0.007s sys 0.000s
I20260810 07:52:56.289737 380 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbccbecf884768b3b171d9bd901b9b (DEFAULT_TABLE table=default.Table_b3adcec668f7473687d358c9b80c10f4 [id=b23efee8dd99419d9308ef2d5a907409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.291018 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbccbecf884768b3b171d9bd901b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.290876 373 tablet_service.cc:1511] Processing CreateTablet for tablet 10341e269a934612bb0cf6ef29818a39 (DEFAULT_TABLE table=default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.292094 373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10341e269a934612bb0cf6ef29818a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.293251 375 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:56.294440 382 tablet_service.cc:1511] Processing CreateTablet for tablet a89cdc44a8c64ccd83fd30855798f87b (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.295476 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89cdc44a8c64ccd83fd30855798f87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.301452 384 tablet_service.cc:1511] Processing CreateTablet for tablet 7eeabaae541548afab4a3ff5ab388035 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.302613 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eeabaae541548afab4a3ff5ab388035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.316311 1071 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:56.259706 1691 log.cc:927] Time spent T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260810 07:52:56.318488 1437 raft_consensus.cc:2243] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:56.328433 1437 raft_consensus.cc:2272] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:56.216708 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 20) took 1259 ms (client timeout 1000 ms). Trace:
W20260810 07:52:56.330839 32464 rpcz_store.cc:269] 0810 07:52:54.957181 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.957338 (+ 157us) service_pool.cc:224] Handling call
0810 07:52:56.216634 (+1259296us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":18}
W20260810 07:52:56.232290 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260810 07:52:56.296900 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 69c05d6e49204eedbb305f66dfa2316c (table default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]) successfully deleted
I20260810 07:52:56.234086 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_11b5cd303ac54cd7ac544914b0d3bfac" } new_table_name: "default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e" modify_external_catalogs: true
I20260810 07:52:56.337924 1437 ts_tablet_manager.cc:1916] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:56.340678 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 5195424dd98f444caa75e2ca98d09e6d (table default.Table_94eb441ee1024892b34ab0ec641b16ff [id=a49a08c7a7954b87aab561894073bf0d]) successfully deleted
W20260810 07:52:56.344048 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 2c171d3358e44ae2962ab3c4c377b0f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c171d3358e44ae2962ab3c4c377b0f1 already in progress: creating tablet
I20260810 07:52:56.349303 1071 tablet_bootstrap.cc:654] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:56.353435 385 tablet_service.cc:1511] Processing CreateTablet for tablet 213fa4838fe847de87091cfba97eaedc (DEFAULT_TABLE table=default.Table_5c9348ac7cc14fb1842cb6bd08652cae [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.354789 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213fa4838fe847de87091cfba97eaedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.354662 375 tablet_service.cc:1511] Processing CreateTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c (DEFAULT_TABLE table=default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.355948 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7000d25f3bfe4c99be54a0f4789cc96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.235539 32461 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_97132925a8444a3d9c702a3ae7cbb209" } modify_external_catalogs: true
I20260810 07:52:56.357229 32461 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_97132925a8444a3d9c702a3ae7cbb209" } modify_external_catalogs: true
I20260810 07:52:56.357604 376 tablet_service.cc:1511] Processing CreateTablet for tablet fc12205a097c496cba5b777b98f27914 (DEFAULT_TABLE table=default.Table_0dae8ea1aefd4618b2dddfc4e459f367 [id=f26d526a9f1949df98cb9ff907fbfe73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.358825 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc12205a097c496cba5b777b98f27914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.229586 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 2c171d3358e44ae2962ab3c4c377b0f1 (table default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]) successfully deleted
W20260810 07:52:56.365247 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 135685us
I20260810 07:52:56.367799 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet c7c136422ffe4bb2b4f77ab081acd19f (table default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]) successfully deleted
W20260810 07:52:56.371610 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 because the tablet was not found. No further retry: Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:56.261114 1686 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:56.265771 1047 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:56.374584 1047 ts_tablet_manager.cc:1403] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.570s user 0.022s sys 0.005s
I20260810 07:52:56.286044 644 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:56.287040 645 tablet_service.cc:1511] Processing CreateTablet for tablet a89cdc44a8c64ccd83fd30855798f87b (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.287357 639 tablet_service.cc:1511] Processing CreateTablet for tablet 7eeabaae541548afab4a3ff5ab388035 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.287997 642 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbccbecf884768b3b171d9bd901b9b (DEFAULT_TABLE table=default.Table_b3adcec668f7473687d358c9b80c10f4 [id=b23efee8dd99419d9308ef2d5a907409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.379379 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89cdc44a8c64ccd83fd30855798f87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.382510 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbccbecf884768b3b171d9bd901b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.384600 1047 raft_consensus.cc:359] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.347474 646 tablet_service.cc:1511] Processing CreateTablet for tablet 10341e269a934612bb0cf6ef29818a39 (DEFAULT_TABLE table=default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.385291 1047 raft_consensus.cc:385] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.353479 633 tablet_service.cc:1511] Processing CreateTablet for tablet 213fa4838fe847de87091cfba97eaedc (DEFAULT_TABLE table=default.Table_5c9348ac7cc14fb1842cb6bd08652cae [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.385565 1047 raft_consensus.cc:740] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:56.355777 631 tablet_service.cc:1511] Processing CreateTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c (DEFAULT_TABLE table=default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.386209 1047 consensus_queue.cc:260] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.362226 630 tablet_service.cc:1511] Processing CreateTablet for tablet fc12205a097c496cba5b777b98f27914 (DEFAULT_TABLE table=default.Table_0dae8ea1aefd4618b2dddfc4e459f367 [id=f26d526a9f1949df98cb9ff907fbfe73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.385041 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eeabaae541548afab4a3ff5ab388035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.391420 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10341e269a934612bb0cf6ef29818a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.398454 1047 ts_tablet_manager.cc:1434] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260810 07:52:56.399300 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7000d25f3bfe4c99be54a0f4789cc96c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:56.400686 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 168403us
I20260810 07:52:56.408420 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213fa4838fe847de87091cfba97eaedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.412912 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc12205a097c496cba5b777b98f27914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.418043 1047 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:56.428151 1047 tablet_bootstrap.cc:654] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:56.442476 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0098aaa7be644c9885167dc1ee4929b5 has no consensus state
I20260810 07:52:56.206233 1694 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:56.445221 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:56.445820 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:52:56.459707 792 consensus_peers.cc:597] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:56.306522 638 tablet_service.cc:1511] Processing CreateTablet for tablet f8415b6376a740388487f7ae4aebea52 (DEFAULT_TABLE table=default.Table_b041641ad99146e8ac57b26f842b8976 [id=0ab295f37b5e482482053f67fe0719a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.467543 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8415b6376a740388487f7ae4aebea52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.470731 32467 catalog_manager.cc:5719] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:56.488372 1671 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.488998 1671 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.489600 382 tablet_service.cc:1511] Processing CreateTablet for tablet 8718f998623e4f1c8b7b8be815e184a3 (DEFAULT_TABLE table=default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.490705 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718f998623e4f1c8b7b8be815e184a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.499213 1555 raft_consensus.cc:493] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.500300 1064 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:56.499940 1555 raft_consensus.cc:515] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.500815 1064 ts_tablet_manager.cc:1403] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.693s user 0.008s sys 0.006s
I20260810 07:52:56.505571 1555 leader_election.cc:290] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:56.506856 309 leader_election.cc:343] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:52:56.517578 644 tablet_service.cc:1511] Processing CreateTablet for tablet 8718f998623e4f1c8b7b8be815e184a3 (DEFAULT_TABLE table=default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.518692 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718f998623e4f1c8b7b8be815e184a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.520990 1671 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:56.522308 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:56.523131 895 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:56.524632 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:56.525408 411 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:56.524956 574 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:56.528805 1600 raft_consensus.cc:2804] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:52:56.529290 1600 raft_consensus.cc:493] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:56.529652 1600 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:56.535454 1555 raft_consensus.cc:493] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.535918 1555 raft_consensus.cc:515] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.539664 1555 leader_election.cc:290] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:56.541179 309 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
W20260810 07:52:56.542680 308 leader_election.cc:343] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:56.543200 308 leader_election.cc:304] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:56.530745 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c136422ffe4bb2b4f77ab081acd19f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:56.553622 308 leader_election.cc:304] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:56.558121 1555 raft_consensus.cc:2749] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:56.559594 1555 raft_consensus.cc:2749] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:56.560508 792 consensus_peers.cc:597] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:56.580199 1047 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:56.519872 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 09712cfebab34e56ba844e7aa0e8d152 (table default.Table_db3c7e150df447dab749a578605e8d4c [id=5fe5c44770974b8eb4f3b55096d25eeb]). Expected version 1 got 0
I20260810 07:52:56.580964 1047 ts_tablet_manager.cc:1403] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.163s user 0.022s sys 0.024s
I20260810 07:52:56.580554 1655 consensus_queue.cc:1048] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:56.581960 32467 catalog_manager.cc:5719] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:56.594540 1047 raft_consensus.cc:359] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.595551 1047 raft_consensus.cc:385] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.595919 1047 raft_consensus.cc:740] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:52:56.600080 635 consensus_meta.cc:287] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Time spent flushing consensus metadata: real 0.514s user 0.002s sys 0.004s
I20260810 07:52:56.600713 1047 consensus_queue.cc:260] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:56.372588 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 1ba026e367ea487fb0dc8aef5d5db606 because the tablet was not found. No further retry: Not found: Tablet not found: 1ba026e367ea487fb0dc8aef5d5db606
W20260810 07:52:56.603330 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 230752us
W20260810 07:52:56.604131 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because the tablet was not found. No further retry: Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
W20260810 07:52:56.605304 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 69c05d6e49204eedbb305f66dfa2316c because the tablet was not found. No further retry: Not found: Tablet not found: 69c05d6e49204eedbb305f66dfa2316c
W20260810 07:52:56.605937 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet acf740de7abb40bfabfc3a7859cc159b because the tablet was not found. No further retry: Not found: Tablet not found: acf740de7abb40bfabfc3a7859cc159b
I20260810 07:52:56.606158 1047 ts_tablet_manager.cc:1434] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
I20260810 07:52:56.607021 1047 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:56.613051 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_4ec9847b17c44e009c6162f1417d5a13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:56.615015 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ec9847b17c44e009c6162f1417d5a13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.614604 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a26f61efd56a4f57b935e0a12c6d7b2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:56.614936 371 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff (DEFAULT_TABLE table=default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:56.617705 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a26f61efd56a4f57b935e0a12c6d7b2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.618410 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe822aa30b54fb091fd62ed8b6e22ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.625281 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b7bfd0d718e746c9b7fc82dba4a4c555"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:56.628365 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260810 07:52:56.636878 1263 raft_consensus.cc:2243] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:56.638482 642 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff (DEFAULT_TABLE table=default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:56.645164 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:56.652168 377 consensus_meta.cc:287] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent flushing consensus metadata: real 0.561s user 0.007s sys 0.000s
I20260810 07:52:56.653708 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe822aa30b54fb091fd62ed8b6e22ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.661396 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:52:56.662055 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:52:56.675449 869 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbccbecf884768b3b171d9bd901b9b (DEFAULT_TABLE table=default.Table_b3adcec668f7473687d358c9b80c10f4 [id=b23efee8dd99419d9308ef2d5a907409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.675635 868 tablet_service.cc:1511] Processing CreateTablet for tablet a89cdc44a8c64ccd83fd30855798f87b (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.676505 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 5195424dd98f444caa75e2ca98d09e6d with delete_type TABLET_DATA_DELETED (replaced by tablet 62352629fce54deebaa507f58ff908e6) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.677347 862 tablet_service.cc:1511] Processing CreateTablet for tablet 7eeabaae541548afab4a3ff5ab388035 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.679469 872 tablet_service.cc:1511] Processing CreateTablet for tablet 10341e269a934612bb0cf6ef29818a39 (DEFAULT_TABLE table=default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.679664 866 tablet_service.cc:1511] Processing CreateTablet for tablet 213fa4838fe847de87091cfba97eaedc (DEFAULT_TABLE table=default.Table_5c9348ac7cc14fb1842cb6bd08652cae [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.681533 32467 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_956f0c9ce19f4e929b17f81669a8e797 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 2 got 0
I20260810 07:52:56.682492 32470 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 62352629fce54deebaa507f58ff908e6
I20260810 07:52:56.682200 860 tablet_service.cc:1511] Processing CreateTablet for tablet 8718f998623e4f1c8b7b8be815e184a3 (DEFAULT_TABLE table=default.Table_8880a03671844a10931425b1c422e0da [id=1083d105acd24b249933790d21f43ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.683121 1671 raft_consensus.cc:493] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.683434 864 tablet_service.cc:1511] Processing CreateTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c (DEFAULT_TABLE table=default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.683607 1671 raft_consensus.cc:515] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.685369 871 tablet_service.cc:1511] Processing CreateTablet for tablet fc12205a097c496cba5b777b98f27914 (DEFAULT_TABLE table=default.Table_0dae8ea1aefd4618b2dddfc4e459f367 [id=f26d526a9f1949df98cb9ff907fbfe73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.685724 1671 leader_election.cc:290] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:56.687207 1671 raft_consensus.cc:493] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.687671 1671 raft_consensus.cc:515] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.687580 859 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff (DEFAULT_TABLE table=default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:56.689273 574 leader_election.cc:343] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:52:56.691414 1671 leader_election.cc:290] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:56.691581 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:56.692133 896 raft_consensus.cc:2468] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:56.693315 574 leader_election.cc:304] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:56.694134 1671 raft_consensus.cc:2804] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:56.694499 1671 raft_consensus.cc:493] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:56.694900 1671 raft_consensus.cc:3060] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:56.639603 1263 raft_consensus.cc:2272] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:56.711072 1263 ts_tablet_manager.cc:1916] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:56.712666 1729 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:52:56.754869 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "449359c8e9fa4756ba149f3dce6e0ce7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:56.756539 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:56.781294 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 09712cfebab34e56ba844e7aa0e8d152
I20260810 07:52:56.785423 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 09712cfebab34e56ba844e7aa0e8d152 deleted and replaced by tablet e36780fb709f427f872f3e58df7b7e81
I20260810 07:52:56.787024 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 1 got 0
I20260810 07:52:56.787803 32469 catalog_manager.cc:5719] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:56.795614 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:56.727027 1047 tablet_bootstrap.cc:654] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:56.757659 573 leader_election.cc:343] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:56.800431 573 leader_election.cc:304] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
W20260810 07:52:56.801599 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 43984.7us
I20260810 07:52:56.802551 1667 raft_consensus.cc:2749] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:56.787133 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:56.814025 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b9d12e0451244cc2967c89a27688e75d" } new_table_name: "default.Table_19a08db6b4c641dc84fd05c3020827f4" modify_external_catalogs: true
I20260810 07:52:56.819039 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_11aa8b64aab44d1fb8a173833ac51e11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:56.717211 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbccbecf884768b3b171d9bd901b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.717404 1064 raft_consensus.cc:359] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.828361 1064 raft_consensus.cc:385] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.742107 1265 ts_tablet_manager.cc:1929] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:56.757220 1730 consensus_queue.cc:1048] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260810 07:52:56.828877 1265 log.cc:1199] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/b2bfffdd01524eada0a85c1a525325a0
I20260810 07:52:56.829675 1265 ts_tablet_manager.cc:1950] T b2bfffdd01524eada0a85c1a525325a0 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:56.789300 856 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:56.828649 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213fa4838fe847de87091cfba97eaedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.833642 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe822aa30b54fb091fd62ed8b6e22ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.828668 1064 raft_consensus.cc:740] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:56.834434 1064 consensus_queue.cc:260] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.836519 1064 ts_tablet_manager.cc:1434] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.335s user 0.005s sys 0.001s
I20260810 07:52:56.837486 1064 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:56.841948 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 98dec9b072ac49959181003cb78567af has no consensus state
I20260810 07:52:56.852999 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_956f0c9ce19f4e929b17f81669a8e797 [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 2 got 0
W20260810 07:52:56.857647 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11aa8b64aab44d1fb8a173833ac51e11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.839814 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10341e269a934612bb0cf6ef29818a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.878582 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89cdc44a8c64ccd83fd30855798f87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.879397 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc12205a097c496cba5b777b98f27914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.883229 1071 tablet_bootstrap.cc:492] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:56.883848 1071 ts_tablet_manager.cc:1403] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.568s user 0.015s sys 0.002s
I20260810 07:52:56.879026 32469 catalog_manager.cc:5719] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:52:56.893200 32461 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 4c13f622081847f88082489de32d58df
I20260810 07:52:56.894853 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_1899e690e1fd4821a2930f385431fafb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:56.896391 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1899e690e1fd4821a2930f385431fafb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.897370 377 tablet_service.cc:1511] Processing CreateTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 (DEFAULT_TABLE table=default.Table_870a1aff5f614bebbde025da24dedfa8 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.901436 32460 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_db3c7e150df447dab749a578605e8d4c" } modify_external_catalogs: true
I20260810 07:52:56.901954 32460 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_db3c7e150df447dab749a578605e8d4c" } modify_external_catalogs: true
I20260810 07:52:56.903461 635 tablet_service.cc:1511] Processing CreateTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 (DEFAULT_TABLE table=default.Table_870a1aff5f614bebbde025da24dedfa8 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:56.907330 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e7b0a1f1654e43f69eac56f97c009155"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:56.910055 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7b0a1f1654e43f69eac56f97c009155 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.857287 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c8825a5b51f241a1b6539a796438bbb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:56.929028 1437 ts_tablet_manager.cc:1929] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:56.932993 1719 raft_consensus.cc:493] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:56.933745 1719 raft_consensus.cc:515] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.937247 1719 leader_election.cc:290] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:56.949050 308 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
W20260810 07:52:56.960985 373 consensus_meta.cc:287] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent flushing consensus metadata: real 0.641s user 0.005s sys 0.003s
I20260810 07:52:56.967978 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 5195424dd98f444caa75e2ca98d09e6d from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:56.973953 1437 log.cc:1199] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/e9a1bcc922db45ad9f5f0edac6c7b5d5
I20260810 07:52:56.974844 1437 ts_tablet_manager.cc:1950] T e9a1bcc922db45ad9f5f0edac6c7b5d5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:56.978770 1071 raft_consensus.cc:359] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:56.912249 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8825a5b51f241a1b6539a796438bbb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:56.994278 1071 raft_consensus.cc:385] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:56.996692 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef32f7951f8640e9a13b069816b1cdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:56.996743 1071 raft_consensus.cc:740] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:56.996246 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 96) took 2492 ms. Trace:
I20260810 07:52:56.982295 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef32f7951f8640e9a13b069816b1cdd5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:57.011046 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 98dec9b072ac49959181003cb78567af has no consensus state
I20260810 07:52:56.793327 1671 raft_consensus.cc:515] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:56.793643 1552 consensus_queue.cc:1048] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:56.794457 1600 raft_consensus.cc:515] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:57.015208 573 consensus_peers.cc:597] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:52:57.015596 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 155523us
W20260810 07:52:57.016878 573 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:57.017488 1600 leader_election.cc:290] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:57.020015 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:57.020927 411 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:57.021690 1265 tablet_replica.cc:333] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
W20260810 07:52:57.025130 573 consensus_peers.cc:597] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:57.021020 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eeabaae541548afab4a3ff5ab388035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:57.029064 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7000d25f3bfe4c99be54a0f4789cc96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:57.029003 856 tablet_service.cc:1511] Processing CreateTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 (DEFAULT_TABLE table=default.Table_870a1aff5f614bebbde025da24dedfa8 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:57.030592 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef32f7951f8640e9a13b069816b1cdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:57.030097 1071 consensus_queue.cc:260] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.033279 411 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 2.
I20260810 07:52:57.038740 1047 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:57.039260 1047 ts_tablet_manager.cc:1403] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.432s user 0.009s sys 0.013s
I20260810 07:52:57.041774 573 leader_election.cc:304] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:57.042522 1552 raft_consensus.cc:2804] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:52:57.043109 1552 raft_consensus.cc:697] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 2 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:57.013159 1437 rpcz_store.cc:276] 0810 07:52:54.503925 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.504046 (+ 121us) service_pool.cc:224] Handling call
0810 07:52:56.277502 (+1773456us) ts_tablet_manager.cc:1151] Deleting tablet e9a1bcc922db45ad9f5f0edac6c7b5d5
0810 07:52:56.277553 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:56.730599 (+453046us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:56.928875 (+198276us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:56.996183 (+ 67308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39851,"tablet-delete.queue_time_us":1772718}
I20260810 07:52:57.036185 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:57.037381 309 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:52:57.044890 309 leader_election.cc:304] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:57.045305 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet e9a1bcc922db45ad9f5f0edac6c7b5d5 (table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]) successfully deleted
I20260810 07:52:57.044163 1552 consensus_queue.cc:237] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:57.047289 573 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:57.049173 1437 tablet_replica.cc:333] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:57.049081 32464 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d6e2c1a2cec54b099ba095449ee1b5a3" } new_table_name: "default.Table_776ba5427fa541f4b2885108ee933c25" modify_external_catalogs: true
I20260810 07:52:57.049912 1437 raft_consensus.cc:2243] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.054718 1719 raft_consensus.cc:2749] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:57.054669 1047 raft_consensus.cc:359] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.057278 1071 ts_tablet_manager.cc:1434] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.173s user 0.004s sys 0.003s
I20260810 07:52:57.066692 1437 raft_consensus.cc:2272] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.066988 1265 raft_consensus.cc:2243] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.068650 1437 ts_tablet_manager.cc:1916] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.070232 1071 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.053042 1600 consensus_queue.cc:1048] T 7c075fdf1055477db9f2d56324f12a8a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:57.081127 1047 raft_consensus.cc:385] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.103927 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_128834a6bbbe4f6da862be15162b8562" } new_table_name: "default.Table_514fab926ff74afcbc734cb171022edf" modify_external_catalogs: true
I20260810 07:52:57.104264 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a26f61efd56a4f57b935e0a12c6d7b2e" } modify_external_catalogs: true
W20260810 07:52:57.113404 32455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 127958us
I20260810 07:52:57.117923 1071 tablet_bootstrap.cc:654] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.134632 1071 tablet_bootstrap.cc:492] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.135164 1071 ts_tablet_manager.cc:1403] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.004s
I20260810 07:52:57.136677 32460 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet e36780fb709f427f872f3e58df7b7e81
I20260810 07:52:57.137998 1071 raft_consensus.cc:359] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.140384 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a26f61efd56a4f57b935e0a12c6d7b2e" } modify_external_catalogs: true
I20260810 07:52:57.141964 1071 raft_consensus.cc:385] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.142323 1071 raft_consensus.cc:740] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.141607 1437 ts_tablet_manager.cc:1929] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.143144 1437 log.cc:1199] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:57.147215 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0da354d9c37742bc843f199cb929c391"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:57.145059 1071 consensus_queue.cc:260] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.143889 1437 ts_tablet_manager.cc:1950] T 95fcae5b2e5448bf89b1c0587f46ad40 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:52:57.149084 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0da354d9c37742bc843f199cb929c391 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:57.154762 1071 ts_tablet_manager.cc:1434] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260810 07:52:57.098656 1671 leader_election.cc:290] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:57.162442 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_868b99bda3b04eafab388ae9cbf490fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:57.164517 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_868b99bda3b04eafab388ae9cbf490fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:57.169697 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 97) took 2608 ms. Trace:
I20260810 07:52:57.170068 1437 rpcz_store.cc:276] 0810 07:52:54.561402 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.561519 (+ 117us) service_pool.cc:224] Handling call
0810 07:52:57.049006 (+2487487us) ts_tablet_manager.cc:1151] Deleting tablet 95fcae5b2e5448bf89b1c0587f46ad40
0810 07:52:57.049051 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.111001 (+ 61950us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.141471 (+ 30470us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.169629 (+ 28158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16085,"tablet-delete.queue_time_us":2486890}
I20260810 07:52:57.171576 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 98) took 2609 ms. Trace:
I20260810 07:52:57.171962 1437 rpcz_store.cc:276] 0810 07:52:54.562111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.562241 (+ 130us) service_pool.cc:224] Handling call
0810 07:52:57.171289 (+2609048us) ts_tablet_manager.cc:1151] Deleting tablet 95fcae5b2e5448bf89b1c0587f46ad40
0810 07:52:57.171331 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.171522 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2608499}
I20260810 07:52:57.173959 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 95fcae5b2e5448bf89b1c0587f46ad40 (table default.Table_452dd2adccb043c685e5eb8af81bd4db [id=c01b9df1b08d4fe5a4dbbab56d704c4f]) successfully deleted
W20260810 07:52:57.174782 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 95fcae5b2e5448bf89b1c0587f46ad40 because the tablet was not found. No further retry: Not found: Tablet not found: 95fcae5b2e5448bf89b1c0587f46ad40
I20260810 07:52:57.097431 1047 raft_consensus.cc:740] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.177645 1071 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.183517 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 99) took 2620 ms. Trace:
I20260810 07:52:57.183857 1437 rpcz_store.cc:276] 0810 07:52:54.563479 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.563605 (+ 126us) service_pool.cc:224] Handling call
0810 07:52:57.183265 (+2619660us) ts_tablet_manager.cc:1151] Deleting tablet d397310bcfd2470e95fc4b0654749aa4
0810 07:52:57.183309 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.183457 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2619151}
W20260810 07:52:57.185119 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet d397310bcfd2470e95fc4b0654749aa4 because the tablet was not found. No further retry: Not found: Tablet not found: d397310bcfd2470e95fc4b0654749aa4
I20260810 07:52:57.188874 1437 tablet_replica.cc:333] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:57.190845 1071 tablet_bootstrap.cc:654] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.189883 1552 raft_consensus.cc:493] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:57.193312 1552 raft_consensus.cc:515] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.198102 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718f998623e4f1c8b7b8be815e184a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:57.200943 1437 raft_consensus.cc:2243] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.201404 1437 raft_consensus.cc:2272] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.203011 1437 ts_tablet_manager.cc:1916] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.205077 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a9ab67614949bb93925144c4675f01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:57.205579 896 raft_consensus.cc:3060] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:52:57.210826 1265 raft_consensus.cc:2272] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.227360 896 raft_consensus.cc:2468] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 2.
I20260810 07:52:57.177552 1047 consensus_queue.cc:260] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.249426 1047 ts_tablet_manager.cc:1434] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.210s user 0.004s sys 0.004s
I20260810 07:52:57.243976 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 1 got 0
I20260810 07:52:57.251027 32467 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a] alter complete (version 1)
I20260810 07:52:57.252434 32463 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d] alter complete (version 3)
W20260810 07:52:57.255880 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 33) took 1520 ms (client timeout 1000 ms). Trace:
I20260810 07:52:57.256247 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:52:57.256335 32463 rpcz_store.cc:269] 0810 07:52:55.735201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.780317 (+ 45116us) service_pool.cc:224] Handling call
0810 07:52:57.255781 (+1475464us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:55.984510 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:55.984987 (+ 477us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:55.985016 (+ 29us) write_op.cc:435] Acquired schema lock
0810 07:52:55.985032 (+ 16us) tablet.cc:662] Decoding operations
0810 07:52:55.985329 (+ 297us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:55.985374 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:55.985397 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:55.985555 (+ 158us) tablet.cc:701] Row locks acquired
0810 07:52:55.985571 (+ 16us) write_op.cc:260] PREPARE: finished
0810 07:52:55.985694 (+ 123us) write_op.cc:270] Start()
0810 07:52:55.985780 (+ 86us) write_op.cc:276] Timestamp: P: 1786348375985652 usec, L: 0
0810 07:52:55.985799 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:52:55.986351 (+ 552us) log.cc:844] Serialized 356 byte log entry
0810 07:52:56.147050 (+160699us) op_driver.cc:464] REPLICATION: finished
0810 07:52:56.162222 (+ 15172us) write_op.cc:301] APPLY: starting
0810 07:52:56.162302 (+ 80us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:56.162409 (+ 107us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:56.162526 (+ 117us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:56.162736 (+ 210us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:56.163055 (+ 319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:56.163102 (+ 47us) write_op.cc:312] APPLY: finished
0810 07:52:56.163277 (+ 175us) log.cc:844] Serialized 23 byte log entry
0810 07:52:56.163429 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:56.163657 (+ 228us) write_op.cc:454] Released schema lock
0810 07:52:56.163690 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:56.181854 (+ 18164us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:56.181891 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":14973,"apply.run_cpu_time_us":1627,"apply.run_wall_time_us":19569,"num_ops":1,"prepare.queue_time_us":27196,"prepare.run_cpu_time_us":2112,"prepare.run_wall_time_us":2119,"raft.queue_time_us":6770,"raft.run_cpu_time_us":872,"raft.run_wall_time_us":16448,"replication_time_us":161149}]]}
I20260810 07:52:57.276924 32463 master_service.cc:960] ReplaceTablet: received request to replace tablet 54a3bd8369634e4bbfdf8920454357bc from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:57.288360 1071 tablet_bootstrap.cc:492] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.288847 1071 ts_tablet_manager.cc:1403] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.111s user 0.011s sys 0.003s
I20260810 07:52:57.291661 1071 raft_consensus.cc:359] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.292191 1437 ts_tablet_manager.cc:1929] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.292467 1071 raft_consensus.cc:385] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.292881 1071 raft_consensus.cc:740] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.292563 1437 log.cc:1199] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/c3675c5bae7a42769223ef5b2fc433b3
I20260810 07:52:57.304949 1437 ts_tablet_manager.cc:1950] T c3675c5bae7a42769223ef5b2fc433b3 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:57.305039 1071 consensus_queue.cc:260] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.307606 1071 ts_tablet_manager.cc:1434] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260810 07:52:57.308583 1071 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.306064 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 101) took 2622 ms. Trace:
I20260810 07:52:57.310648 1437 rpcz_store.cc:276] 0810 07:52:54.683649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.683762 (+ 113us) service_pool.cc:224] Handling call
0810 07:52:57.188698 (+2504936us) ts_tablet_manager.cc:1151] Deleting tablet c3675c5bae7a42769223ef5b2fc433b3
0810 07:52:57.188764 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.215490 (+ 26726us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.292075 (+ 76585us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.305985 (+ 13910us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2504368}
I20260810 07:52:57.311973 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 102) took 2344 ms. Trace:
I20260810 07:52:57.312271 1437 rpcz_store.cc:276] 0810 07:52:54.967711 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.975015 (+ 7304us) service_pool.cc:224] Handling call
0810 07:52:57.311639 (+2336624us) ts_tablet_manager.cc:1151] Deleting tablet c1faed468bfa4d1ba5d92c0b9b194781
0810 07:52:57.311679 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.311920 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2335980}
I20260810 07:52:57.314868 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 103) took 2306 ms. Trace:
W20260810 07:52:57.315009 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1faed468bfa4d1ba5d92c0b9b194781 already in progress: creating tablet
I20260810 07:52:57.315156 1437 rpcz_store.cc:276] 0810 07:52:55.008403 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.008561 (+ 158us) service_pool.cc:224] Handling call
0810 07:52:57.314539 (+2305978us) ts_tablet_manager.cc:1151] Deleting tablet 449359c8e9fa4756ba149f3dce6e0ce7
0810 07:52:57.314577 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.314816 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2305346}
I20260810 07:52:57.316219 1071 tablet_bootstrap.cc:654] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.316444 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 104) took 2264 ms. Trace:
W20260810 07:52:57.316607 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 449359c8e9fa4756ba149f3dce6e0ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449359c8e9fa4756ba149f3dce6e0ce7 already in progress: creating tablet
I20260810 07:52:57.316781 1437 rpcz_store.cc:276] 0810 07:52:55.052106 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.052592 (+ 486us) service_pool.cc:224] Handling call
0810 07:52:57.316132 (+2263540us) ts_tablet_manager.cc:1151] Deleting tablet cf45a33485e9463d89b5ce8affa0bd22
0810 07:52:57.316169 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.316390 (+ 221us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2262926}
I20260810 07:52:57.317754 1437 tablet_replica.cc:333] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:57.318516 1437 raft_consensus.cc:2243] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.319072 1437 raft_consensus.cc:2272] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:57.321269 1729 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.261s user 0.001s sys 0.000s
W20260810 07:52:57.328289 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf45a33485e9463d89b5ce8affa0bd22 already in progress: creating tablet
I20260810 07:52:57.338886 1071 tablet_bootstrap.cc:492] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.339306 1071 ts_tablet_manager.cc:1403] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.000s
I20260810 07:52:57.340968 1437 ts_tablet_manager.cc:1916] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.341861 1071 raft_consensus.cc:359] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.342504 1071 raft_consensus.cc:385] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.342777 1071 raft_consensus.cc:740] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.343586 1071 consensus_queue.cc:260] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.348655 1071 ts_tablet_manager.cc:1434] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260810 07:52:57.352360 1071 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.370939 1071 tablet_bootstrap.cc:654] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.399912 1064 tablet_bootstrap.cc:654] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.429845 1437 ts_tablet_manager.cc:1929] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260810 07:52:57.430243 1437 log.cc:1199] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/5195424dd98f444caa75e2ca98d09e6d
I20260810 07:52:57.430967 1437 ts_tablet_manager.cc:1950] T 5195424dd98f444caa75e2ca98d09e6d P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:57.437295 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 105) took 1901 ms. Trace:
I20260810 07:52:57.437618 1437 rpcz_store.cc:276] 0810 07:52:55.536336 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.536496 (+ 160us) service_pool.cc:224] Handling call
0810 07:52:57.317612 (+1781116us) ts_tablet_manager.cc:1151] Deleting tablet 5195424dd98f444caa75e2ca98d09e6d
0810 07:52:57.317649 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.382092 (+ 64443us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.429699 (+ 47607us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.437226 (+ 7527us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1780292}
I20260810 07:52:57.445600 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 110) took 1575 ms. Trace:
I20260810 07:52:57.451548 1437 rpcz_store.cc:276] 0810 07:52:55.870291 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.870462 (+ 171us) service_pool.cc:224] Handling call
0810 07:52:57.445244 (+1574782us) ts_tablet_manager.cc:1151] Deleting tablet 2c171d3358e44ae2962ab3c4c377b0f1
0810 07:52:57.445290 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.445542 (+ 252us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1574125}
W20260810 07:52:57.452889 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 2c171d3358e44ae2962ab3c4c377b0f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c171d3358e44ae2962ab3c4c377b0f1 already in progress: creating tablet
I20260810 07:52:57.465219 1437 tablet_replica.cc:333] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:57.465993 1437 raft_consensus.cc:2243] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260810 07:52:57.466563 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c292cb975c504ad8abf03e913140c7a9 consensus state has no leader
I20260810 07:52:57.468060 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 20a9ab67614949bb93925144c4675f01 from {username='slave'} at 127.0.0.1:43806
W20260810 07:52:57.470211 32467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 23) took 1537 ms (client timeout 1000 ms). Trace:
W20260810 07:52:57.482017 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 21) took 1425 ms (client timeout 1000 ms). Trace:
W20260810 07:52:57.470543 32467 rpcz_store.cc:269] 0810 07:52:55.933143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.935930 (+ 2787us) service_pool.cc:224] Handling call
0810 07:52:56.752842 (+816912us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:56.752871 (+ 29us) write_op.cc:454] Released schema lock
0810 07:52:57.470144 (+717273us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:56.701920 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:56.702299 (+ 379us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:56.702322 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:52:56.702334 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:56.702612 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:56.702648 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:56.702669 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:56.702940 (+ 271us) tablet.cc:701] Row locks acquired
0810 07:52:56.702953 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:56.703062 (+ 109us) write_op.cc:270] Start()
0810 07:52:56.703134 (+ 72us) write_op.cc:276] Timestamp: P: 1786348376702994 usec, L: 0
0810 07:52:56.703149 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:56.703699 (+ 550us) log.cc:844] Serialized 913 byte log entry
0810 07:52:56.713790 (+ 10091us) op_driver.cc:464] REPLICATION: finished
0810 07:52:56.714399 (+ 609us) write_op.cc:301] APPLY: starting
0810 07:52:56.714478 (+ 79us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:56.714637 (+ 159us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:56.714835 (+ 198us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:56.715178 (+ 343us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:56.715453 (+ 275us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:56.715530 (+ 77us) write_op.cc:312] APPLY: finished
0810 07:52:56.715793 (+ 263us) log.cc:844] Serialized 29 byte log entry
0810 07:52:56.716001 (+ 208us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:56.716111 (+ 110us) write_op.cc:454] Released schema lock
0810 07:52:56.716160 (+ 49us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:52:56.973604 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:56.974440 (+ 836us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:56.974472 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:52:56.974495 (+ 23us) tablet.cc:662] Decoding operations
0810 07:52:56.974786 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:56.974825 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:56.974846 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:56.976640 (+ 1794us) tablet.cc:701] Row locks acquired
0810 07:52:56.976656 (+ 16us) write_op.cc:260] PREPARE: finished
0810 07:52:56.976782 (+ 126us) write_op.cc:270] Start()
0810 07:52:56.976859 (+ 77us) write_op.cc:276] Timestamp: P: 1786348376976691 usec, L: 0
0810 07:52:56.976877 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:52:56.977449 (+ 572us) log.cc:844] Serialized 356 byte log entry
0810 07:52:57.114937 (+137488us) op_driver.cc:464] REPLICATION: finished
0810 07:52:57.126807 (+ 11870us) write_op.cc:301] APPLY: starting
0810 07:52:57.126884 (+ 77us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:57.126998 (+ 114us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:57.127108 (+ 110us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:57.127319 (+ 211us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:57.127493 (+ 174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:57.127550 (+ 57us) write_op.cc:312] APPLY: finished
0810 07:52:57.127755 (+ 205us) log.cc:844] Serialized 23 byte log entry
0810 07:52:57.127945 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.128049 (+ 104us) write_op.cc:454] Released schema lock
0810 07:52:57.128084 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:57.128403 (+ 319us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.128432 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":217,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":435,"apply.run_cpu_time_us":2032,"apply.run_wall_time_us":2032,"num_ops":2,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":2099,"prepare.run_wall_time_us":2099,"raft.queue_time_us":1194,"raft.run_cpu_time_us":676,"raft.run_wall_time_us":741,"replication_time_us":10534}],["op",{"apply.queue_time_us":11689,"apply.run_cpu_time_us":1528,"apply.run_wall_time_us":1529,"num_ops":1,"prepare.queue_time_us":73090,"prepare.run_cpu_time_us":2344,"prepare.run_wall_time_us":5195,"raft.queue_time_us":235,"raft.run_cpu_time_us":606,"raft.run_wall_time_us":607,"replication_time_us":137938}]]}
W20260810 07:52:57.482374 32469 rpcz_store.cc:269] 0810 07:52:56.057159 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.057314 (+ 155us) service_pool.cc:224] Handling call
0810 07:52:57.481942 (+1424628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:57.128171 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:57.169036 (+ 40865us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:57.169064 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:52:57.169076 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:57.169382 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:57.169421 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:57.169443 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:57.169677 (+ 234us) tablet.cc:701] Row locks acquired
0810 07:52:57.169690 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:57.169813 (+ 123us) write_op.cc:270] Start()
0810 07:52:57.169896 (+ 83us) write_op.cc:276] Timestamp: P: 1786348377169748 usec, L: 0
0810 07:52:57.169910 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:52:57.170587 (+ 677us) log.cc:844] Serialized 913 byte log entry
0810 07:52:57.214509 (+ 43922us) op_driver.cc:464] REPLICATION: finished
0810 07:52:57.214868 (+ 359us) write_op.cc:301] APPLY: starting
0810 07:52:57.214926 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:57.215069 (+ 143us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:57.215183 (+ 114us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:57.215466 (+ 283us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:57.215617 (+ 151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:57.215662 (+ 45us) write_op.cc:312] APPLY: finished
0810 07:52:57.215859 (+ 197us) log.cc:844] Serialized 29 byte log entry
0810 07:52:57.217479 (+ 1620us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.217563 (+ 84us) write_op.cc:454] Released schema lock
0810 07:52:57.217600 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:57.233948 (+ 16348us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.233988 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":125810,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":212,"apply.run_cpu_time_us":1509,"apply.run_wall_time_us":2950,"num_ops":2,"prepare.queue_time_us":199810,"prepare.run_cpu_time_us":2311,"prepare.run_wall_time_us":105747,"raft.queue_time_us":99,"raft.run_cpu_time_us":860,"raft.run_wall_time_us":860,"replication_time_us":44491}]]}
I20260810 07:52:57.489089 1437 raft_consensus.cc:2272] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.495638 1437 ts_tablet_manager.cc:1916] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.249657 1263 ts_tablet_manager.cc:1929] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260810 07:52:57.289104 573 consensus_peers.cc:597] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:52:57.511176 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 222155us
I20260810 07:52:57.479519 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 5195424dd98f444caa75e2ca98d09e6d with delete_type TABLET_DATA_DELETED (replaced by tablet 62352629fce54deebaa507f58ff908e6) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:57.511188 1263 log.cc:1199] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:52:57.518128 1263 ts_tablet_manager.cc:1950] T c7c136422ffe4bb2b4f77ab081acd19f P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:57.534477 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 135) took 1609 ms. Trace:
I20260810 07:52:57.535140 1263 rpcz_store.cc:276] 0810 07:52:55.925450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.927685 (+ 2235us) service_pool.cc:224] Handling call
0810 07:52:56.243495 (+315810us) ts_tablet_manager.cc:1151] Deleting tablet c7c136422ffe4bb2b4f77ab081acd19f
0810 07:52:56.243531 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.143229 (+899698us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.249524 (+106295us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.534382 (+284858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211843,"tablet-delete.queue_time_us":206479}
I20260810 07:52:57.537647 1263 tablet_replica.cc:333] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:57.537689 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet c7c136422ffe4bb2b4f77ab081acd19f (table default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]) successfully deleted
I20260810 07:52:57.539314 1263 raft_consensus.cc:2243] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.540060 1263 raft_consensus.cc:2272] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.542641 1263 ts_tablet_manager.cc:1916] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.546082 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:57.570993 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3d59e31692ec40109b7e69c59019dd83" } modify_external_catalogs: true
I20260810 07:52:57.571611 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3d59e31692ec40109b7e69c59019dd83" } modify_external_catalogs: true
I20260810 07:52:57.435207 1265 ts_tablet_manager.cc:1916] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.550482 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c292cb975c504ad8abf03e913140c7a9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:57.574553 894 raft_consensus.cc:3060] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:57.576851 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f79cf4b4853c4f2d8147136e256dc1aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:57.578521 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f79cf4b4853c4f2d8147136e256dc1aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:57.583766 894 raft_consensus.cc:2468] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:52:57.548429 573 leader_election.cc:343] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:57.597088 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 48728.9us
I20260810 07:52:57.600788 1719 raft_consensus.cc:493] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:57.601315 1719 raft_consensus.cc:515] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.603382 1719 leader_election.cc:290] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:57.606355 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:57.607123 1437 ts_tablet_manager.cc:1929] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.607676 1437 log.cc:1199] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:52:57.609481 1437 ts_tablet_manager.cc:1950] T c7c136422ffe4bb2b4f77ab081acd19f P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:52:57.610211 32464 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8779af4159eb48da894e6a92c2c4a6f2 consensus state has no leader
I20260810 07:52:57.612152 1071 tablet_bootstrap.cc:492] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.612658 1071 ts_tablet_manager.cc:1403] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.263s user 0.081s sys 0.072s
I20260810 07:52:57.615650 1071 raft_consensus.cc:359] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.616300 1071 raft_consensus.cc:385] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.616421 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 111) took 1686 ms. Trace:
I20260810 07:52:57.622082 1437 rpcz_store.cc:276] 0810 07:52:55.925853 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.041212 (+115359us) service_pool.cc:224] Handling call
0810 07:52:57.465053 (+1423841us) ts_tablet_manager.cc:1151] Deleting tablet c7c136422ffe4bb2b4f77ab081acd19f
0810 07:52:57.465097 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.535536 (+ 70439us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.606865 (+ 71329us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.612335 (+ 5470us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1371890}
I20260810 07:52:57.616572 1071 raft_consensus.cc:740] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.635120 1437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 116) took 1346 ms. Trace:
I20260810 07:52:57.635928 1071 consensus_queue.cc:260] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.635686 1437 rpcz_store.cc:276] 0810 07:52:56.289156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.293083 (+ 3927us) service_pool.cc:224] Handling call
0810 07:52:57.632997 (+1339914us) ts_tablet_manager.cc:1151] Deleting tablet b2bfffdd01524eada0a85c1a525325a0
0810 07:52:57.633044 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.635037 (+ 1993us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1339275}
I20260810 07:52:57.642561 1437 tablet_replica.cc:333] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:52:57.555253 1047 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:57.645704 1437 raft_consensus.cc:2243] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:57.651641 1437 raft_consensus.cc:2272] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.653637 1437 ts_tablet_manager.cc:1916] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.665894 1552 leader_election.cc:290] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:57.677140 862 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:57.677723 860 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:57.679041 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:57.679805 892 raft_consensus.cc:2468] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:57.681603 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:57.692795 309 leader_election.cc:304] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:57.699425 411 raft_consensus.cc:3060] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:57.701537 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet c7c136422ffe4bb2b4f77ab081acd19f (table default.Table_f571ac3561af4ca9b497338a81e1f9a9 [id=ad284bd00dd7415b92f06251f2d66211]) successfully deleted
I20260810 07:52:57.705628 413 raft_consensus.cc:3060] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:57.712540 860 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:57.715384 868 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55008 (request call id 127) took 1040 ms. Trace:
I20260810 07:52:57.638087 1071 ts_tablet_manager.cc:1434] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260810 07:52:57.715709 1064 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:57.725157 1064 ts_tablet_manager.cc:1403] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.888s user 0.013s sys 0.008s
I20260810 07:52:57.725466 868 rpcz_store.cc:276] 0810 07:52:56.675037 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.675126 (+ 89us) service_pool.cc:224] Handling call
0810 07:52:56.720826 (+ 45700us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:56.721570 (+ 744us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:57.602218 (+880648us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.666831 (+ 64613us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:57.715309 (+ 48478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:52:57.726164 1071 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.741043 1719 raft_consensus.cc:2749] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:57.733536 1753 raft_consensus.cc:493] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:57.734978 1064 raft_consensus.cc:359] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.750154 1064 raft_consensus.cc:385] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.750504 1064 raft_consensus.cc:740] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.750219 1753 raft_consensus.cc:515] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.751196 1064 consensus_queue.cc:260] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.752386 1753 leader_election.cc:290] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:57.754130 574 consensus_peers.cc:597] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:52:57.757917 574 leader_election.cc:304] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:57.758754 1263 ts_tablet_manager.cc:1929] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.759128 1263 log.cc:1199] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:52:57.759788 1263 ts_tablet_manager.cc:1950] T 92e2678ab165486dbbb69c7c183a7fa4 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:57.760077 1671 raft_consensus.cc:2804] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:57.760542 1671 raft_consensus.cc:697] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:57.760990 1071 tablet_bootstrap.cc:654] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.761691 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 139) took 1724 ms. Trace:
I20260810 07:52:57.762959 1064 ts_tablet_manager.cc:1434] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260810 07:52:57.763878 1064 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:57.761302 1671 consensus_queue.cc:237] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.764638 574 leader_election.cc:304] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:57.765859 1600 raft_consensus.cc:2804] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:57.766368 1600 raft_consensus.cc:493] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:57.766701 1600 raft_consensus.cc:3060] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:57.791798 1263 rpcz_store.cc:276] 0810 07:52:56.037530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.037663 (+ 133us) service_pool.cc:224] Handling call
0810 07:52:57.537415 (+1499752us) ts_tablet_manager.cc:1151] Deleting tablet 92e2678ab165486dbbb69c7c183a7fa4
0810 07:52:57.537480 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.654319 (+116839us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.758630 (+104311us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.761638 (+ 3008us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1499167}
I20260810 07:52:57.793256 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 92e2678ab165486dbbb69c7c183a7fa4 (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]) successfully deleted
I20260810 07:52:57.798211 1437 ts_tablet_manager.cc:1929] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.803016 1047 tablet_bootstrap.cc:654] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.809146 1437 log.cc:1199] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/4af5109e338844139444bb7111619195
I20260810 07:52:57.810114 1437 ts_tablet_manager.cc:1950] T 4af5109e338844139444bb7111619195 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:57.813797 1263 tablet_replica.cc:333] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:57.802462 1770 raft_consensus.cc:493] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:57.814764 1263 raft_consensus.cc:2243] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:57.804791 1071 tablet_bootstrap.cc:492] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.815821 1263 pending_rounds.cc:70] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Trying to abort 1 pending ops.
I20260810 07:52:57.816000 1071 ts_tablet_manager.cc:1403] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.090s user 0.012s sys 0.000s
I20260810 07:52:57.816159 1263 pending_rounds.cc:77] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882948825915392 op_type: NO_OP noop_request { }
I20260810 07:52:57.817416 1263 raft_consensus.cc:2889] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:52:57.818246 32462 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 5baaf994466c4073a721be4b0fcb33e2
I20260810 07:52:57.818679 1071 raft_consensus.cc:359] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.819332 1071 raft_consensus.cc:385] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.814949 1770 raft_consensus.cc:515] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.822551 1064 tablet_bootstrap.cc:654] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.819597 1071 raft_consensus.cc:740] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:52:57.825424 308 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
W20260810 07:52:57.828433 309 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
I20260810 07:52:57.831588 309 leader_election.cc:304] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:57.832422 1719 raft_consensus.cc:2749] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:57.825943 1770 leader_election.cc:290] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:57.836948 32463 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 54a3bd8369634e4bbfdf8920454357bc
I20260810 07:52:57.838376 32463 catalog_manager.cc:6807] ReplaceTablet: tablet 54a3bd8369634e4bbfdf8920454357bc deleted and replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90
I20260810 07:52:57.840101 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:57.840920 1263 raft_consensus.cc:2272] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:57.841054 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:57.843281 1263 ts_tablet_manager.cc:1916] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:57.843199 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:57.840358 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:57.848961 871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55008 (request call id 133) took 1168 ms. Trace:
I20260810 07:52:57.849301 871 rpcz_store.cc:276] 0810 07:52:56.680968 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.684478 (+ 3510us) service_pool.cc:224] Handling call
0810 07:52:56.726107 (+ 41629us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:56.726186 (+ 79us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:57.567054 (+840868us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.848477 (+281423us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:57.848871 (+ 394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:52:57.854044 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:57.863622 1770 raft_consensus.cc:493] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:57.864080 1770 raft_consensus.cc:515] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.865767 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 4af5109e338844139444bb7111619195 (table default.Table_0741e2ea30224c6180516e0791c77331 [id=1898c24eab7d4065b04abf4e98141671]) successfully deleted
W20260810 07:52:57.868901 309 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 4ec00a494816409684205c7f4435a079
I20260810 07:52:57.869825 1071 consensus_queue.cc:260] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:57.874532 792 leader_election.cc:343] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:57.876502 1071 ts_tablet_manager.cc:1434] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.060s user 0.006s sys 0.000s
I20260810 07:52:57.878758 872 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55008 (request call id 130) took 1200 ms. Trace:
I20260810 07:52:57.879099 872 rpcz_store.cc:276] 0810 07:52:56.678605 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.678725 (+ 120us) service_pool.cc:224] Handling call
0810 07:52:56.721253 (+ 42528us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:56.721348 (+ 95us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:57.656642 (+935294us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.878338 (+221696us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:57.878698 (+ 360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:52:57.879837 1770 leader_election.cc:290] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:57.885373 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54a3bd8369634e4bbfdf8920454357bc already in progress: creating tablet
W20260810 07:52:57.892896 792 leader_election.cc:343] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:57.893392 792 leader_election.cc:304] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:57.894515 1753 raft_consensus.cc:2749] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:57.909552 1600 raft_consensus.cc:515] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:57.911031 308 leader_election.cc:343] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 4ec00a494816409684205c7f4435a079
I20260810 07:52:57.911473 308 leader_election.cc:304] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:57.912329 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:52:57.913829 573 leader_election.cc:343] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:57.913970 1265 ts_tablet_manager.cc:1929] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:57.914328 1265 log.cc:1199] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/09712cfebab34e56ba844e7aa0e8d152
I20260810 07:52:57.915205 1265 ts_tablet_manager.cc:1950] T 09712cfebab34e56ba844e7aa0e8d152 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:57.917073 1047 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:57.917577 1047 ts_tablet_manager.cc:1403] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.363s user 0.014s sys 0.000s
I20260810 07:52:57.916958 411 raft_consensus.cc:3060] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:57.920379 1770 raft_consensus.cc:2749] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:57.922045 1071 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.933110 32460 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_a448e407c7ec4922ab6bf6b5bb01b6f5" } new_table_name: "default.Table_227a31b168824c84a470faf1069ad0fa" modify_external_catalogs: true
I20260810 07:52:57.936542 1265 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 136) took 1147 ms. Trace:
I20260810 07:52:57.934597 1064 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:57.937252 1064 ts_tablet_manager.cc:1403] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.174s user 0.012s sys 0.005s
I20260810 07:52:57.939013 1071 tablet_bootstrap.cc:654] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:57.939778 1064 raft_consensus.cc:359] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.940469 1064 raft_consensus.cc:385] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.940812 1064 raft_consensus.cc:740] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.941700 1064 consensus_queue.cc:260] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.945875 1047 raft_consensus.cc:359] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.947088 1047 raft_consensus.cc:385] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.947166 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_fb6ddd83c8f44109b5a1be3203096778" } new_table_name: "default.Table_f26b16b9c60448cd8f984bf3f106a0f2" modify_external_catalogs: true
I20260810 07:52:57.947425 1047 raft_consensus.cc:740] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.948287 1047 consensus_queue.cc:260] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.954052 1047 ts_tablet_manager.cc:1434] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260810 07:52:57.954381 1265 rpcz_store.cc:276] 0810 07:52:56.788648 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.789014 (+ 366us) service_pool.cc:224] Handling call
0810 07:52:57.021521 (+232507us) ts_tablet_manager.cc:1151] Deleting tablet 09712cfebab34e56ba844e7aa0e8d152
0810 07:52:57.021570 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.787734 (+766164us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.913845 (+126111us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.936482 (+ 22637us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218725,"tablet-delete.queue_time_us":191070}
W20260810 07:52:57.957116 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet b2bfffdd01524eada0a85c1a525325a0 because the tablet was not found. No further retry: Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0
W20260810 07:52:57.957952 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c7c136422ffe4bb2b4f77ab081acd19f because the tablet was not found. No further retry: Not found: Tablet not found: c7c136422ffe4bb2b4f77ab081acd19f
W20260810 07:52:57.958954 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 2c171d3358e44ae2962ab3c4c377b0f1 because the tablet was not found. No further retry: Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
W20260810 07:52:57.960598 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54a3bd8369634e4bbfdf8920454357bc already in progress: creating tablet
I20260810 07:52:57.970963 413 raft_consensus.cc:1275] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260810 07:52:57.977066 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_8e034abdc8d141e4bc8c80614ba099f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:57.981753 1064 ts_tablet_manager.cc:1434] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
I20260810 07:52:57.984278 1071 tablet_bootstrap.cc:492] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:57.984853 1071 ts_tablet_manager.cc:1403] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.004s
I20260810 07:52:57.985363 1064 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:57.987251 1071 raft_consensus.cc:359] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:57.987857 1071 raft_consensus.cc:385] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:57.988131 1071 raft_consensus.cc:740] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:57.991065 1064 tablet_bootstrap.cc:654] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.005080 1071 consensus_queue.cc:260] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.010332 1071 ts_tablet_manager.cc:1434] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.025s user 0.004s sys 0.001s
I20260810 07:52:58.013983 1071 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:57.965965 1790 tablet.cc:1724] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:58.021687 1600 leader_election.cc:290] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:58.022883 1047 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:52:58.025168 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e034abdc8d141e4bc8c80614ba099f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:57.973363 1263 ts_tablet_manager.cc:1929] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:57.975637 1671 consensus_queue.cc:1048] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:58.037086 1263 log.cc:1199] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/b2bfffdd01524eada0a85c1a525325a0
I20260810 07:52:58.037863 1263 ts_tablet_manager.cc:1950] T b2bfffdd01524eada0a85c1a525325a0 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:58.039858 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 143) took 1754 ms. Trace:
I20260810 07:52:58.028672 1071 tablet_bootstrap.cc:654] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:58.049834 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56563.7us
I20260810 07:52:58.040217 1263 rpcz_store.cc:276] 0810 07:52:56.285775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.285887 (+ 112us) service_pool.cc:224] Handling call
0810 07:52:57.810074 (+1524187us) ts_tablet_manager.cc:1151] Deleting tablet b2bfffdd01524eada0a85c1a525325a0
0810 07:52:57.813654 (+ 3580us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:57.911428 (+ 97774us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:57.973238 (+ 61810us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:58.039802 (+ 66564us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1433167}
I20260810 07:52:58.058519 1263 tablet_replica.cc:333] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:58.059628 1263 raft_consensus.cc:2243] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:58.060292 1263 raft_consensus.cc:2272] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:58.064308 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c292cb975c504ad8abf03e913140c7a9 consensus state has no leader
I20260810 07:52:58.068485 1071 tablet_bootstrap.cc:492] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.069020 1071 ts_tablet_manager.cc:1403] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.001s
I20260810 07:52:58.072222 1071 raft_consensus.cc:359] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.073843 1071 raft_consensus.cc:385] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.074331 1071 raft_consensus.cc:740] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.075093 1071 consensus_queue.cc:260] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.077121 1071 ts_tablet_manager.cc:1434] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260810 07:52:58.084669 1071 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.096621 1071 tablet_bootstrap.cc:654] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.028671 1047 tablet_bootstrap.cc:654] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.101140 1263 ts_tablet_manager.cc:1916] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:52:58.119151 32462 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55aaf296555b43f9a27f2926054c8d4b has no consensus state
I20260810 07:52:58.124207 1064 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:58.124634 1064 ts_tablet_manager.cc:1403] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.139s user 0.001s sys 0.008s
I20260810 07:52:58.125694 1071 tablet_bootstrap.cc:492] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.126108 1071 ts_tablet_manager.cc:1403] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.006s
I20260810 07:52:58.127079 1064 raft_consensus.cc:359] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.127707 1064 raft_consensus.cc:385] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.127974 1064 raft_consensus.cc:740] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.128288 1729 tablet.cc:1724] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:58.132041 1071 raft_consensus.cc:359] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.132699 1071 raft_consensus.cc:385] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.133059 1071 raft_consensus.cc:740] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.134133 1071 consensus_queue.cc:260] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.138170 1071 ts_tablet_manager.cc:1434] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260810 07:52:58.139081 1071 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.122341 1552 raft_consensus.cc:493] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.141409 1552 raft_consensus.cc:515] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.144373 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:58.144871 1552 leader_election.cc:290] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:58.145866 573 leader_election.cc:343] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.148100 1071 tablet_bootstrap.cc:654] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.149880 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:58.150499 894 raft_consensus.cc:3060] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.157681 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:58.157903 892 raft_consensus.cc:1275] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260810 07:52:58.164281 574 leader_election.cc:343] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.164680 574 leader_election.cc:304] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:58.165722 1552 raft_consensus.cc:2749] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:58.168535 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0752f7e6b5e84c50bb807a28862c4e6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:58.169735 1064 consensus_queue.cc:260] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.170861 1071 tablet_bootstrap.cc:492] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.171281 1071 ts_tablet_manager.cc:1403] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.003s
I20260810 07:52:58.172135 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_826c5b5c4ab84216af2be3ae4ce22024" } modify_external_catalogs: true
I20260810 07:52:58.173061 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_826c5b5c4ab84216af2be3ae4ce22024" } modify_external_catalogs: true
W20260810 07:52:58.178270 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0752f7e6b5e84c50bb807a28862c4e6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:58.173581 1071 raft_consensus.cc:359] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.178764 1064 ts_tablet_manager.cc:1434] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.054s user 0.004s sys 0.001s
I20260810 07:52:58.178857 1071 raft_consensus.cc:385] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.179291 1071 raft_consensus.cc:740] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.180006 1071 consensus_queue.cc:260] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.183475 1071 ts_tablet_manager.cc:1434] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260810 07:52:58.184381 1071 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.197319 1064 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:58.211912 1071 tablet_bootstrap.cc:654] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.215430 1047 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:58.215883 1047 ts_tablet_manager.cc:1403] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.193s user 0.015s sys 0.000s
I20260810 07:52:58.093966 1790 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 154) took 1196 ms. Trace:
I20260810 07:52:58.219740 1790 rpcz_store.cc:276] 0810 07:52:56.897111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.900639 (+ 3528us) service_pool.cc:224] Handling call
0810 07:52:58.093896 (+1193257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:56.903764 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:52:56.904332 (+ 568us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:52:56.904365 (+ 33us) alter_schema_op.cc:66] Acquired schema lock
0810 07:52:56.904503 (+ 138us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:52:56.904666 (+ 163us) alter_schema_op.cc:128] START. Timestamp: 1786348376904554
0810 07:52:56.904712 (+ 46us) op_driver.cc:348] REPLICATION: starting
0810 07:52:57.018972 (+114260us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540009d420. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22bc72c951 00007f22bc740900 00007f22bc7c9f22 00007f22bc7bf347 00007f22bc7bf184 00007f22bcc4ef61 00007f22bcc4e738 00007f22bcc533d1 00007f22bcc53389 00007f22bcc53319 00007f22bcc532e1
0810 07:52:57.019718 (+ 746us) log.cc:844] Serialized 173 byte log entry
0810 07:52:57.921296 (+901578us) op_driver.cc:464] REPLICATION: finished
0810 07:52:57.965843 (+ 44547us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:52:58.016825 (+ 50982us) log.cc:844] Serialized 5 byte log entry
0810 07:52:58.088927 (+ 72102us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:58.093082 (+ 4155us) log.cc:844] Serialized 14 byte log entry
0810 07:52:58.093770 (+ 688us) alter_schema_op.cc:73] Released schema lock
0810 07:52:58.093800 (+ 30us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":44337,"prepare.queue_time_us":447,"prepare.run_cpu_time_us":12891,"prepare.run_wall_time_us":138457,"raft.queue_time_us":152,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":1016465,"spinlock_wait_cycles":250178048,"thread_start_us":113651,"threads_started":2,"wal-append.queue_time_us":20430}]]}
I20260810 07:52:58.225055 411 raft_consensus.cc:3060] T 7c075fdf1055477db9f2d56324f12a8a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.236076 1071 tablet_bootstrap.cc:492] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.236608 1071 ts_tablet_manager.cc:1403] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.001s
I20260810 07:52:58.239041 1071 raft_consensus.cc:359] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.239682 1071 raft_consensus.cc:385] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.239939 1071 raft_consensus.cc:740] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.240795 1064 tablet_bootstrap.cc:654] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.240549 1071 consensus_queue.cc:260] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.242985 1071 ts_tablet_manager.cc:1434] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:52:58.244088 1071 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.247953 1770 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.248382 1770 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.249388 894 raft_consensus.cc:2468] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:58.251075 1770 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:58.251377 1071 tablet_bootstrap.cc:654] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.257411 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3d1903c2423c4def8e00c8316402d0dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:58.264500 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:58.267846 1753 raft_consensus.cc:493] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.268402 1753 raft_consensus.cc:515] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.270283 32469 master_service.cc:960] ReplaceTablet: received request to replace tablet 7eeabaae541548afab4a3ff5ab388035 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:58.270614 1753 leader_election.cc:290] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.223486 1047 raft_consensus.cc:359] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.281316 1047 raft_consensus.cc:385] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.281653 1047 raft_consensus.cc:740] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.281692 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:58.283035 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.282357 1047 consensus_queue.cc:260] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.286259 1047 ts_tablet_manager.cc:1434] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20260810 07:52:58.287318 1047 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:52:58.289553 792 leader_election.cc:343] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:52:58.299598 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d1903c2423c4def8e00c8316402d0dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:58.300194 1071 tablet_bootstrap.cc:492] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.300863 1071 ts_tablet_manager.cc:1403] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.057s user 0.019s sys 0.001s
I20260810 07:52:58.304636 1071 raft_consensus.cc:359] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.305553 1071 raft_consensus.cc:385] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.305840 1071 raft_consensus.cc:740] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.306476 1071 consensus_queue.cc:260] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.309355 1071 ts_tablet_manager.cc:1434] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260810 07:52:58.310271 1071 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.320413 1071 tablet_bootstrap.cc:654] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.247084 1263 ts_tablet_manager.cc:1929] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:58.252890 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.325001 1263 log.cc:1199] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/09712cfebab34e56ba844e7aa0e8d152
I20260810 07:52:58.277141 1600 consensus_queue.cc:1048] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:58.326229 308 leader_election.cc:304] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.279733 574 leader_election.cc:304] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:58.328258 1770 raft_consensus.cc:2749] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:58.327425 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 47794us
I20260810 07:52:58.330472 1770 raft_consensus.cc:493] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.331162 1770 raft_consensus.cc:515] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.333062 1600 raft_consensus.cc:2804] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:58.334684 1600 raft_consensus.cc:697] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:58.337081 1071 tablet_bootstrap.cc:492] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.337528 1071 ts_tablet_manager.cc:1403] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.002s
I20260810 07:52:58.337713 1719 raft_consensus.cc:493] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.335071 1263 ts_tablet_manager.cc:1950] T 09712cfebab34e56ba844e7aa0e8d152 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:58.338165 1719 raft_consensus.cc:515] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.340154 1263 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 153) took 1556 ms. Trace:
I20260810 07:52:58.341023 1071 raft_consensus.cc:359] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.341657 1071 raft_consensus.cc:385] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.341892 1071 raft_consensus.cc:740] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.342566 1071 consensus_queue.cc:260] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.336544 1600 consensus_queue.cc:237] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.340507 1719 leader_election.cc:290] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:58.344843 309 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0
I20260810 07:52:58.346745 1071 ts_tablet_manager.cc:1434] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260810 07:52:58.347589 308 leader_election.cc:343] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0
I20260810 07:52:58.348135 308 leader_election.cc:304] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.348887 1071 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.356442 1071 tablet_bootstrap.cc:654] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.359570 1064 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:58.360057 1064 ts_tablet_manager.cc:1403] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.163s user 0.006s sys 0.004s
W20260810 07:52:58.360255 309 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:52:58.361486 1770 leader_election.cc:290] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:58.362739 1770 raft_consensus.cc:2749] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:58.363940 1064 raft_consensus.cc:359] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.364563 1064 raft_consensus.cc:385] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.364861 1064 raft_consensus.cc:740] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.340490 1263 rpcz_store.cc:276] 0810 07:52:56.784286 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.786696 (+ 2410us) service_pool.cc:224] Handling call
0810 07:52:58.058291 (+1271595us) ts_tablet_manager.cc:1151] Deleting tablet 09712cfebab34e56ba844e7aa0e8d152
0810 07:52:58.058353 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:52:58.212977 (+154624us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:58.246941 (+ 33964us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:58.340087 (+ 93146us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1254498}
W20260810 07:52:58.367868 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 5195424dd98f444caa75e2ca98d09e6d because the tablet was not found. No further retry: Not found: Tablet not found: 5195424dd98f444caa75e2ca98d09e6d
W20260810 07:52:58.369490 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54a3bd8369634e4bbfdf8920454357bc already in progress: creating tablet
I20260810 07:52:58.388559 1071 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.389189 1071 ts_tablet_manager.cc:1403] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.013s
W20260810 07:52:58.320988 792 leader_election.cc:343] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.390810 792 leader_election.cc:304] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.392194 1753 raft_consensus.cc:2749] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:58.392871 1071 raft_consensus.cc:359] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:58.393288 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 72377us
I20260810 07:52:58.393554 1071 raft_consensus.cc:385] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.393816 1071 raft_consensus.cc:740] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.394228 1064 consensus_queue.cc:260] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.394419 1071 consensus_queue.cc:260] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.396466 1047 tablet_bootstrap.cc:654] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.398346 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_a6f9405788014781b2dc4ed324067e30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:58.404150 1071 ts_tablet_manager.cc:1434] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260810 07:52:58.405118 1071 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:58.410770 1064 ts_tablet_manager.cc:1434] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.050s user 0.002s sys 0.006s
I20260810 07:52:58.414650 1071 tablet_bootstrap.cc:654] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:58.421195 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6f9405788014781b2dc4ed324067e30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:52:58.422722 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 75178.8us
W20260810 07:52:58.425312 308 leader_election.cc:343] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:52:58.425833 308 leader_election.cc:304] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.427414 1821 raft_consensus.cc:2749] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:58.434621 1064 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:58.487164 1047 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:58.487674 1047 ts_tablet_manager.cc:1403] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.201s user 0.012s sys 0.004s
I20260810 07:52:58.493466 411 raft_consensus.cc:3060] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.494271 413 raft_consensus.cc:3060] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.503562 1753 raft_consensus.cc:493] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.504199 1753 raft_consensus.cc:515] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.507891 1753 leader_election.cc:290] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.527212 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.527769 1047 raft_consensus.cc:359] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.528421 1047 raft_consensus.cc:385] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:52:58.529024 792 leader_election.cc:343] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.532826 1064 tablet_bootstrap.cc:654] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.528677 1047 raft_consensus.cc:740] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.538288 1047 consensus_queue.cc:260] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.539628 1753 raft_consensus.cc:493] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.540079 1753 raft_consensus.cc:515] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.542040 1753 leader_election.cc:290] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.542537 414 raft_consensus.cc:3060] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.545804 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:58.548596 792 leader_election.cc:343] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.548563 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.553318 1071 tablet_bootstrap.cc:492] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.553155 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:58.553722 1071 ts_tablet_manager.cc:1403] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.149s user 0.023s sys 0.026s
I20260810 07:52:58.558535 1071 raft_consensus.cc:359] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.559468 1071 raft_consensus.cc:385] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.559737 1071 raft_consensus.cc:740] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.560397 1071 consensus_queue.cc:260] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.563135 1552 consensus_queue.cc:1048] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:58.566037 1071 ts_tablet_manager.cc:1434] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260810 07:52:58.567078 1071 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:52:58.574394 792 leader_election.cc:343] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.575134 792 leader_election.cc:304] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.576092 1753 raft_consensus.cc:2749] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:58.577419 792 leader_election.cc:343] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.577867 792 leader_election.cc:304] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:58.578655 1753 raft_consensus.cc:2749] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:58.582350 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_58600ebaa02642eea48fbf663c45afe2" } new_table_name: "default.Table_71d864751edf4f7786f4aea1d9e5e8ad" modify_external_catalogs: true
I20260810 07:52:58.592706 1047 ts_tablet_manager.cc:1434] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.105s user 0.003s sys 0.004s
I20260810 07:52:58.597723 1071 tablet_bootstrap.cc:654] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.595064 1047 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:58.613142 1064 tablet_bootstrap.cc:492] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:58.613605 1064 ts_tablet_manager.cc:1403] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.179s user 0.016s sys 0.010s
I20260810 07:52:58.616395 1064 raft_consensus.cc:359] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.617103 1064 raft_consensus.cc:385] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.617355 1064 raft_consensus.cc:740] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.618121 1064 consensus_queue.cc:260] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.626929 1064 ts_tablet_manager.cc:1434] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260810 07:52:58.628197 1064 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:58.634419 1064 tablet_bootstrap.cc:654] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.648471 1064 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:58.648926 1064 ts_tablet_manager.cc:1403] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.003s
I20260810 07:52:58.651748 1064 raft_consensus.cc:359] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.652444 1064 raft_consensus.cc:385] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.652801 1064 raft_consensus.cc:740] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.653977 1064 consensus_queue.cc:260] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.660017 1064 ts_tablet_manager.cc:1434] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260810 07:52:58.662384 1064 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:58.671032 1064 tablet_bootstrap.cc:654] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:58.709939 32461 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a04e29d3a49b4bd891be79e0fad6ec51 consensus state has no leader
I20260810 07:52:58.738411 1047 tablet_bootstrap.cc:654] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.746549 1064 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:58.750952 1064 ts_tablet_manager.cc:1403] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.089s user 0.030s sys 0.042s
I20260810 07:52:58.753610 1064 raft_consensus.cc:359] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.602561 1843 tablet.cc:1724] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:52:58.754931 1064 raft_consensus.cc:385] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.755182 1064 raft_consensus.cc:740] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.755793 1064 consensus_queue.cc:260] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.757766 1064 ts_tablet_manager.cc:1434] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:52:58.759214 1753 raft_consensus.cc:493] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.759676 1753 raft_consensus.cc:515] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.761546 1753 leader_election.cc:290] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.763607 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.764199 678 raft_consensus.cc:2468] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:58.765692 792 leader_election.cc:304] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:58.766597 1753 raft_consensus.cc:2804] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:58.766963 1753 raft_consensus.cc:493] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:58.767333 1753 raft_consensus.cc:3060] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.769873 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:58.770538 413 raft_consensus.cc:2468] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:58.771229 1047 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:58.771653 1047 ts_tablet_manager.cc:1403] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.177s user 0.013s sys 0.000s
I20260810 07:52:58.774639 1047 raft_consensus.cc:359] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.775315 1047 raft_consensus.cc:385] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.775583 1047 raft_consensus.cc:740] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.776212 1047 consensus_queue.cc:260] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.779753 1064 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:58.778441 1047 ts_tablet_manager.cc:1434] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260810 07:52:58.790939 1047 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:58.797166 1047 tablet_bootstrap.cc:654] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.798791 414 raft_consensus.cc:3060] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.818915 1600 consensus_queue.cc:1048] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:58.822839 1856 raft_consensus.cc:493] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:58.823321 1856 raft_consensus.cc:515] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.826166 1856 leader_election.cc:290] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.828455 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:58.829176 678 raft_consensus.cc:2468] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:52:58.830880 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:58.832444 792 leader_election.cc:343] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.833130 792 leader_election.cc:304] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:58.833941 1856 raft_consensus.cc:2804] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:58.834352 1856 raft_consensus.cc:493] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:58.834833 1856 raft_consensus.cc:3060] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.846005 1856 raft_consensus.cc:515] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.855685 1071 tablet_bootstrap.cc:492] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:58.856189 1071 ts_tablet_manager.cc:1403] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.289s user 0.009s sys 0.012s
I20260810 07:52:58.858525 1856 leader_election.cc:290] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.860417 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:58.862313 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:58.863101 678 raft_consensus.cc:3060] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:52:58.872068 792 leader_election.cc:343] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:58.874151 678 raft_consensus.cc:2468] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:58.877745 1753 raft_consensus.cc:515] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.889106 792 leader_election.cc:304] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:58.890486 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:58.891131 413 raft_consensus.cc:3060] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.908998 1856 raft_consensus.cc:2804] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:58.909516 1856 raft_consensus.cc:697] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:58.910300 1856 consensus_queue.cc:237] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.918277 1071 raft_consensus.cc:359] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.923394 1071 raft_consensus.cc:385] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:58.924260 1071 raft_consensus.cc:740] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:58.926057 1071 consensus_queue.cc:260] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.942571 1071 ts_tablet_manager.cc:1434] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.086s user 0.004s sys 0.004s
I20260810 07:52:58.943048 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f]). Expected version 1 got 0
I20260810 07:52:58.878247 1064 tablet_bootstrap.cc:654] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:58.969949 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf45a33485e9463d89b5ce8affa0bd22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:52:58.971210 678 raft_consensus.cc:3060] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:58.972023 1753 leader_election.cc:290] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:58.979439 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 7eeabaae541548afab4a3ff5ab388035
I20260810 07:52:58.981222 32469 catalog_manager.cc:6807] ReplaceTablet: tablet 7eeabaae541548afab4a3ff5ab388035 deleted and replaced by tablet 1cb66183211f45d4a756f70c5e331f87
I20260810 07:52:58.985916 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:58.986914 872 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:58.995265 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:52:59.007052 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 3fe822aa30b54fb091fd62ed8b6e22ff
I20260810 07:52:59.010576 642 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.012444 371 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:59.017014 871 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:59.017769 642 tablet_service.cc:1511] Processing CreateTablet for tablet 45c5e309389d402092f59c801923854a (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.018921 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c5e309389d402092f59c801923854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.019177 871 tablet_service.cc:1511] Processing CreateTablet for tablet 45c5e309389d402092f59c801923854a (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.020232 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c5e309389d402092f59c801923854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:58.961166 413 raft_consensus.cc:2468] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:58.969444 1821 raft_consensus.cc:493] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.029596 1552 raft_consensus.cc:493] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.029800 1821 raft_consensus.cc:515] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:58.984006 377 tablet_service.cc:1511] Processing CreateTablet for tablet 45c5e309389d402092f59c801923854a (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:58.985232 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:59.035027 1821 leader_election.cc:290] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:59.035914 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7eeabaae541548afab4a3ff5ab388035 already in progress: creating tablet
W20260810 07:52:59.038281 308 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 09712cfebab34e56ba844e7aa0e8d152
W20260810 07:52:59.040804 309 leader_election.cc:343] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 09712cfebab34e56ba844e7aa0e8d152
W20260810 07:52:58.993897 311 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 48343us
I20260810 07:52:59.041311 309 leader_election.cc:304] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:52:59.043027 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 3fe822aa30b54fb091fd62ed8b6e22ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3fe822aa30b54fb091fd62ed8b6e22ff already in progress: creating tablet
I20260810 07:52:59.044160 1821 raft_consensus.cc:2749] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:59.045313 1821 raft_consensus.cc:493] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.046017 1821 raft_consensus.cc:515] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.048465 1064 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:59.048996 1064 ts_tablet_manager.cc:1403] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.269s user 0.013s sys 0.002s
I20260810 07:52:59.049963 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:59.054379 1821 leader_election.cc:290] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.054606 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57324aac10d42d191f72e2c2343d4ea" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:59.055375 895 raft_consensus.cc:2393] T a57324aac10d42d191f72e2c2343d4ea P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:52:59.056672 309 leader_election.cc:304] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:59.058831 1071 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:59.061548 792 leader_election.cc:304] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:52:59.062758 1753 raft_consensus.cc:2804] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:59.064337 1753 raft_consensus.cc:697] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:52:59.065634 1753 consensus_queue.cc:237] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.061980 1064 raft_consensus.cc:359] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.059274 1821 raft_consensus.cc:3060] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.068367 1064 raft_consensus.cc:385] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.068631 1064 raft_consensus.cc:740] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.074046 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c5e309389d402092f59c801923854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.077164 1064 consensus_queue.cc:260] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.080471 1047 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:59.080933 1047 ts_tablet_manager.cc:1403] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.290s user 0.091s sys 0.148s
I20260810 07:52:59.083384 1047 raft_consensus.cc:359] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.084029 1047 raft_consensus.cc:385] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.084285 1047 raft_consensus.cc:740] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.085173 1047 consensus_queue.cc:260] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.090513 872 tablet_service.cc:1511] Processing CreateTablet for tablet 82edc39a864f49f8be60e4b36ce3d59c (DEFAULT_TABLE table=default.Table_af59b24731ec4cf8a7c32525e4891ee7 [id=9915d36da31542d0b018cd2463bda067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.092397 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82edc39a864f49f8be60e4b36ce3d59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.097025 1552 raft_consensus.cc:515] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.099088 1047 ts_tablet_manager.cc:1434] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.018s user 0.004s sys 0.001s
W20260810 07:52:59.099977 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a04e29d3a49b4bd891be79e0fad6ec51 consensus state has no leader
I20260810 07:52:59.100875 1600 raft_consensus.cc:493] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.101468 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:59.101424 1600 raft_consensus.cc:515] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.102159 895 raft_consensus.cc:2468] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:59.103546 1600 leader_election.cc:290] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.105340 574 leader_election.cc:304] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:59.106065 1600 raft_consensus.cc:2804] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:59.106444 1064 ts_tablet_manager.cc:1434] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.054s user 0.004s sys 0.002s
I20260810 07:52:59.106451 1600 raft_consensus.cc:493] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:59.106910 1600 raft_consensus.cc:3060] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.106804 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:59.107414 895 raft_consensus.cc:2468] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:59.108865 574 leader_election.cc:304] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:59.109582 1671 raft_consensus.cc:2804] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:59.109933 1671 raft_consensus.cc:493] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:59.110249 1671 raft_consensus.cc:3060] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.116982 635 tablet_service.cc:1511] Processing CreateTablet for tablet 82edc39a864f49f8be60e4b36ce3d59c (DEFAULT_TABLE table=default.Table_af59b24731ec4cf8a7c32525e4891ee7 [id=9915d36da31542d0b018cd2463bda067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.117625 1671 raft_consensus.cc:515] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.121302 1600 raft_consensus.cc:515] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:52:59.124315 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 3fe822aa30b54fb091fd62ed8b6e22ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3fe822aa30b54fb091fd62ed8b6e22ff already in progress: creating tablet
W20260810 07:52:59.125319 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7eeabaae541548afab4a3ff5ab388035 already in progress: creating tablet
I20260810 07:52:59.127995 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.128618 414 raft_consensus.cc:2468] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:59.129885 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:52:59.131438 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7eeabaae541548afab4a3ff5ab388035 already in progress: creating tablet
W20260810 07:52:59.133246 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 3fe822aa30b54fb091fd62ed8b6e22ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3fe822aa30b54fb091fd62ed8b6e22ff already in progress: creating tablet
I20260810 07:52:59.133653 382 tablet_service.cc:1511] Processing CreateTablet for tablet 82edc39a864f49f8be60e4b36ce3d59c (DEFAULT_TABLE table=default.Table_af59b24731ec4cf8a7c32525e4891ee7 [id=9915d36da31542d0b018cd2463bda067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.134716 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82edc39a864f49f8be60e4b36ce3d59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.132150 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:59.137072 414 raft_consensus.cc:3060] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.142859 1071 tablet_bootstrap.cc:654] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:59.143901 871 tablet_service.cc:1511] Processing CreateTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.121845 1671 leader_election.cc:290] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.122197 1552 leader_election.cc:290] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.123607 1047 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:59.130841 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82edc39a864f49f8be60e4b36ce3d59c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.152478 1047 tablet_bootstrap.cc:654] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:59.155937 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_1ac5b694f50044beb1c28f2388c02f4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.157244 1600 leader_election.cc:290] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:52:59.161617 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ac5b694f50044beb1c28f2388c02f4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.164990 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8779af4159eb48da894e6a92c2c4a6f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:59.165587 895 raft_consensus.cc:3060] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.145260 631 tablet_service.cc:1511] Processing CreateTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.167610 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0098aaa7be644c9885167dc1ee4929b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.173043 1753 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.173609 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:59.174217 894 raft_consensus.cc:3060] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.178026 1821 raft_consensus.cc:2749] T a57324aac10d42d191f72e2c2343d4ea P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:59.183928 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 20a9ab67614949bb93925144c4675f01
I20260810 07:52:59.185421 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0098aaa7be644c9885167dc1ee4929b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.185564 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 20a9ab67614949bb93925144c4675f01 deleted and replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1
I20260810 07:52:59.185916 32468 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 191) took 1860 ms. Trace:
I20260810 07:52:59.186415 868 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a9ab67614949bb93925144c4675f01 with delete_type TABLET_DATA_DELETED (replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:59.187151 1874 tablet_replica.cc:333] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:59.187791 633 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a9ab67614949bb93925144c4675f01 with delete_type TABLET_DATA_DELETED (replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.187906 1874 raft_consensus.cc:2243] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:59.188405 1874 raft_consensus.cc:2272] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:59.188534 1871 tablet_replica.cc:333] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:59.189360 1871 raft_consensus.cc:2243] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 2 LEADER]: Raft consensus shutting down.
I20260810 07:52:59.190250 1871 raft_consensus.cc:2272] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:59.192296 1871 ts_tablet_manager.cc:1916] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:59.186190 32468 rpcz_store.cc:276] 0810 07:52:57.325168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.467951 (+142783us) service_pool.cc:224] Handling call
0810 07:52:57.570068 (+102117us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ed41 00007f30cf440d37 00007f30cf440cc2 00007f30cf440bf1 00007f30cf440b6c 00007f30cf43fe22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:59.185861 (+1615793us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:58.934057 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:58.934400 (+ 343us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:58.934429 (+ 29us) write_op.cc:435] Acquired schema lock
0810 07:52:58.934441 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:58.934706 (+ 265us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:58.934756 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:58.934777 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:58.934955 (+ 178us) tablet.cc:701] Row locks acquired
0810 07:52:58.934970 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:52:58.935065 (+ 95us) write_op.cc:270] Start()
0810 07:52:58.935134 (+ 69us) write_op.cc:276] Timestamp: P: 1786348378934999 usec, L: 0
0810 07:52:58.935149 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:58.935632 (+ 483us) log.cc:844] Serialized 628 byte log entry
0810 07:52:59.181139 (+245507us) op_driver.cc:464] REPLICATION: finished
0810 07:52:59.181381 (+ 242us) write_op.cc:301] APPLY: starting
0810 07:52:59.181437 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:59.181544 (+ 107us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:59.181677 (+ 133us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:59.182106 (+ 429us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:59.182355 (+ 249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:59.182396 (+ 41us) write_op.cc:312] APPLY: finished
0810 07:52:59.182575 (+ 179us) log.cc:844] Serialized 29 byte log entry
0810 07:52:59.183239 (+ 664us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:59.183305 (+ 66us) write_op.cc:454] Released schema lock
0810 07:52:59.183336 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:59.183583 (+ 247us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:59.183605 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":231149824,"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":1677,"apply.run_wall_time_us":2175,"num_ops":2,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":1839,"prepare.run_wall_time_us":1967,"raft.queue_time_us":75,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":245897}]]}
I20260810 07:52:59.197422 1600 raft_consensus.cc:493] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.197412 895 raft_consensus.cc:2468] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:59.198009 1600 raft_consensus.cc:515] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.200273 868 tablet_service.cc:1511] Processing CreateTablet for tablet f0606ccac5d04aa89d804cf9168018b8 (DEFAULT_TABLE table=default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.201328 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0606ccac5d04aa89d804cf9168018b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.203261 1874 ts_tablet_manager.cc:1916] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:59.173635 1753 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.206550 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.206993 1753 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:52:59.208894 792 leader_election.cc:343] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.213522 894 raft_consensus.cc:2468] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:59.215137 1064 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:59.198774 574 leader_election.cc:304] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:59.225582 574 leader_election.cc:304] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:52:59.207834 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:59.142305 678 raft_consensus.cc:2468] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:52:59.219813 642 tablet_service.cc:1511] Processing CreateTablet for tablet f0606ccac5d04aa89d804cf9168018b8 (DEFAULT_TABLE table=default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:59.227495 792 leader_election.cc:343] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.227905 792 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:59.228556 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0606ccac5d04aa89d804cf9168018b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.229513 1552 raft_consensus.cc:2804] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:59.229997 1552 raft_consensus.cc:697] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:59.230355 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f]). Expected version 1 got 0
I20260810 07:52:59.230783 1552 consensus_queue.cc:237] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.231289 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:52:59.231906 1835 raft_consensus.cc:2804] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:59.232682 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:52:59.234464 1753 raft_consensus.cc:2749] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:52:59.238894 573 leader_election.cc:343] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.242940 1600 leader_election.cc:290] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.242923 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_93e4811656944714b25bb9e0de7256c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.243721 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:52:59.244338 895 raft_consensus.cc:2468] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:52:59.245635 574 leader_election.cc:304] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:52:59.244470 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93e4811656944714b25bb9e0de7256c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.249801 1671 raft_consensus.cc:2804] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:52:59.250233 1671 raft_consensus.cc:493] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:52:59.251076 1671 raft_consensus.cc:3060] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.255450 633 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.258607 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.259104 860 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.260448 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.228443 414 raft_consensus.cc:2468] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:59.273526 1552 raft_consensus.cc:493] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.274000 1552 raft_consensus.cc:515] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.283977 1064 tablet_bootstrap.cc:654] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:59.285373 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:59.287854 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.290153 377 tablet_service.cc:1511] Processing CreateTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.290591 371 tablet_service.cc:1511] Processing CreateTablet for tablet f0606ccac5d04aa89d804cf9168018b8 (DEFAULT_TABLE table=default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.293051 384 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a9ab67614949bb93925144c4675f01 with delete_type TABLET_DATA_DELETED (replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:59.295048 376 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a (DEFAULT_TABLE table=default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.296264 1880 tablet_replica.cc:333] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
W20260810 07:52:59.297834 573 leader_election.cc:343] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.299244 384 tablet_service.cc:1511] Processing CreateTablet for tablet a9be900b701a45728a44bf26350a26ab (DEFAULT_TABLE table=default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.304515 1880 raft_consensus.cc:2243] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:59.305198 1880 raft_consensus.cc:2272] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:59.307996 1880 ts_tablet_manager.cc:1916] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:59.314318 32461 catalog_manager.cc:5719] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:59.321221 1835 raft_consensus.cc:697] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:59.322268 1835 consensus_queue.cc:237] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.326138 862 tablet_service.cc:1511] Processing CreateTablet for tablet a9be900b701a45728a44bf26350a26ab (DEFAULT_TABLE table=default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.327450 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9be900b701a45728a44bf26350a26ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.335379 1047 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:59.336169 1047 ts_tablet_manager.cc:1403] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.213s user 0.010s sys 0.001s
I20260810 07:52:59.339290 1047 raft_consensus.cc:359] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.343565 1071 tablet_bootstrap.cc:492] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:59.344575 1071 ts_tablet_manager.cc:1403] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.286s user 0.010s sys 0.017s
I20260810 07:52:59.316619 32461 catalog_manager.cc:5719] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:52:59.345722 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:52:59.347517 1071 raft_consensus.cc:359] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.348001 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:52:59.348151 1071 raft_consensus.cc:385] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.348408 1071 raft_consensus.cc:740] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.369266 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0098aaa7be644c9885167dc1ee4929b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.373329 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9be900b701a45728a44bf26350a26ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:59.366037 574 leader_election.cc:343] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:52:59.367149 633 tablet_service.cc:1511] Processing CreateTablet for tablet a9be900b701a45728a44bf26350a26ab (DEFAULT_TABLE table=default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.376658 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9be900b701a45728a44bf26350a26ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.378127 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:59.380554 573 leader_election.cc:343] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.376686 1071 consensus_queue.cc:260] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.386044 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0606ccac5d04aa89d804cf9168018b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.392146 1071 ts_tablet_manager.cc:1434] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20260810 07:52:59.397629 1671 raft_consensus.cc:515] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.401541 1071 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:52:59.416185 872 tablet_service.cc:1511] Processing CreateTablet for tablet 45a3fe9686e0445187c77cf41c563c01 (DEFAULT_TABLE table=default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.417526 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a3fe9686e0445187c77cf41c563c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.430943 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_524b25e2037e4ccaa99cbeeca49631f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.431694 1064 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:59.432183 1064 ts_tablet_manager.cc:1403] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.217s user 0.009s sys 0.006s
W20260810 07:52:59.432559 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_524b25e2037e4ccaa99cbeeca49631f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.382061 1047 raft_consensus.cc:385] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.434517 1047 raft_consensus.cc:740] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.435560 1047 consensus_queue.cc:260] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.438606 1047 ts_tablet_manager.cc:1434] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.102s user 0.006s sys 0.000s
I20260810 07:52:59.443293 868 tablet_service.cc:1511] Processing CreateTablet for tablet d301af14299040528cf3a033ff5177f8 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.444633 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d301af14299040528cf3a033ff5177f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.446071 631 tablet_service.cc:1511] Processing CreateTablet for tablet cec929b5f0e14f6586ed50137ecd741b (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.447386 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec929b5f0e14f6586ed50137ecd741b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.447002 1064 raft_consensus.cc:359] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.447664 1064 raft_consensus.cc:385] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.447957 1064 raft_consensus.cc:740] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.362186 1552 leader_election.cc:290] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.421980 635 tablet_service.cc:1511] Processing CreateTablet for tablet 45a3fe9686e0445187c77cf41c563c01 (DEFAULT_TABLE table=default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.454329 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a3fe9686e0445187c77cf41c563c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.458909 1871 ts_tablet_manager.cc:1929] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:52:59.459364 1871 log.cc:1199] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/20a9ab67614949bb93925144c4675f01
I20260810 07:52:59.460215 1871 ts_tablet_manager.cc:1950] T 20a9ab67614949bb93925144c4675f01 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:59.463485 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]) successfully deleted
I20260810 07:52:59.466956 1671 leader_election.cc:290] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.468151 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:52:59.468878 894 raft_consensus.cc:3060] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:52:59.469760 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_749f194454fe43a88634da1983fc5584"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:59.471561 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_749f194454fe43a88634da1983fc5584 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.477515 1047 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:59.478890 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:52:59.479774 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:52:59.480825 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:52:59.480685 642 tablet_service.cc:1511] Processing CreateTablet for tablet d301af14299040528cf3a033ff5177f8 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.486713 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c171d3358e44ae2962ab3c4c377b0f1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.487871 871 tablet_service.cc:1511] Processing CreateTablet for tablet cec929b5f0e14f6586ed50137ecd741b (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.489180 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec929b5f0e14f6586ed50137ecd741b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.490713 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:52:59.492204 1835 consensus_queue.cc:1048] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:52:59.493103 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d301af14299040528cf3a033ff5177f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.448603 1064 consensus_queue.cc:260] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.501533 638 tablet_service.cc:1511] Processing CreateTablet for tablet f6b54db926fa4032b6408d3d309de35b (DEFAULT_TABLE table=default.Table_1899e690e1fd4821a2930f385431fafb [id=0548157d3d3c49cca88607a747dfd017]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:52:59.503441 573 leader_election.cc:343] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.503466 856 tablet_service.cc:1511] Processing CreateTablet for tablet f6b54db926fa4032b6408d3d309de35b (DEFAULT_TABLE table=default.Table_1899e690e1fd4821a2930f385431fafb [id=0548157d3d3c49cca88607a747dfd017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.504088 573 leader_election.cc:304] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:52:59.505250 1835 raft_consensus.cc:2749] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:59.507654 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b54db926fa4032b6408d3d309de35b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:59.508348 573 leader_election.cc:343] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.524183 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b54db926fa4032b6408d3d309de35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.526567 373 tablet_service.cc:1511] Processing CreateTablet for tablet 45a3fe9686e0445187c77cf41c563c01 (DEFAULT_TABLE table=default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.529481 373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a3fe9686e0445187c77cf41c563c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.537225 894 raft_consensus.cc:2468] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:52:59.514204 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:52:59.538733 574 leader_election.cc:304] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:52:59.539777 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:52:59.539700 1671 consensus_queue.cc:1048] T a57324aac10d42d191f72e2c2343d4ea P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:52:59.540602 1835 raft_consensus.cc:2804] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:52:59.542116 375 tablet_service.cc:1511] Processing CreateTablet for tablet d301af14299040528cf3a033ff5177f8 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.543931 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d301af14299040528cf3a033ff5177f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.545229 1835 raft_consensus.cc:697] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:52:59.546561 1064 ts_tablet_manager.cc:1434] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.114s user 0.005s sys 0.003s
I20260810 07:52:59.546536 1835 consensus_queue.cc:237] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.554397 385 tablet_service.cc:1511] Processing CreateTablet for tablet cec929b5f0e14f6586ed50137ecd741b (DEFAULT_TABLE table=default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.555702 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec929b5f0e14f6586ed50137ecd741b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.558159 380 tablet_service.cc:1511] Processing CreateTablet for tablet f6b54db926fa4032b6408d3d309de35b (DEFAULT_TABLE table=default.Table_1899e690e1fd4821a2930f385431fafb [id=0548157d3d3c49cca88607a747dfd017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.567482 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b54db926fa4032b6408d3d309de35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.580530 1071 tablet_bootstrap.cc:654] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:59.585587 1064 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:52:59.594704 644 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe1a943230432dbe533e7b3671172b (DEFAULT_TABLE table=default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.596848 864 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe1a943230432dbe533e7b3671172b (DEFAULT_TABLE table=default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.598197 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe1a943230432dbe533e7b3671172b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.607266 1753 raft_consensus.cc:493] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.607796 1753 raft_consensus.cc:515] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.610369 1753 leader_election.cc:290] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:59.613049 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:52:59.614668 792 leader_election.cc:343] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.615512 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.616500 1064 tablet_bootstrap.cc:654] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:52:59.617197 792 leader_election.cc:343] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.618613 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe1a943230432dbe533e7b3671172b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.618479 792 leader_election.cc:304] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:52:59.619403 1753 raft_consensus.cc:2749] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:59.622960 871 tablet_service.cc:1511] Processing CreateTablet for tablet 98dec9b072ac49959181003cb78567af (DEFAULT_TABLE table=default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.623977 871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98dec9b072ac49959181003cb78567af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.630882 630 tablet_service.cc:1511] Processing CreateTablet for tablet 98dec9b072ac49959181003cb78567af (DEFAULT_TABLE table=default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.632182 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98dec9b072ac49959181003cb78567af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.592062 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0dae8ea1aefd4618b2dddfc4e459f367" } new_table_name: "default.Table_f7f1ce7f0e8848ffa1dc9250b95fa516" modify_external_catalogs: true
I20260810 07:52:59.654528 868 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef (DEFAULT_TABLE table=default.Table_4ec9847b17c44e009c6162f1417d5a13 [id=b70f05a8651b408393dcb8a5cae4507d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.659399 868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.663420 1874 ts_tablet_manager.cc:1929] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:52:59.666517 1874 log.cc:1199] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/20a9ab67614949bb93925144c4675f01
I20260810 07:52:59.671200 1874 ts_tablet_manager.cc:1950] T 20a9ab67614949bb93925144c4675f01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:52:59.679380 1753 raft_consensus.cc:493] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.679872 1753 raft_consensus.cc:515] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.682008 1753 leader_election.cc:290] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:52:59.684295 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:52:59.685810 1071 tablet_bootstrap.cc:492] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:52:59.686220 1071 ts_tablet_manager.cc:1403] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.285s user 0.008s sys 0.004s
W20260810 07:52:59.688277 792 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:52:59.688781 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:52:59.693801 639 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e0a84839427099a9da06e2b970ce (DEFAULT_TABLE table=default.Table_8bed0806196e4e6393b129851ab29404 [id=a4439e8baa3e43d5980744518075a788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.693679 862 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e0a84839427099a9da06e2b970ce (DEFAULT_TABLE table=default.Table_8bed0806196e4e6393b129851ab29404 [id=a4439e8baa3e43d5980744518075a788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.694957 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e0a84839427099a9da06e2b970ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.695209 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e0a84839427099a9da06e2b970ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.701419 32468 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:52:59.703107 32468 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:52:59.703748 32470 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_956f0c9ce19f4e929b17f81669a8e797 [id=3eee7f69234b4dde8cd856bd44a9db70] alter complete (version 2)
I20260810 07:52:59.704273 32467 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 9ae4a28e89714c65b7551f36d1dc78b4
I20260810 07:52:59.706010 32467 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 192) took 2365 ms. Trace:
W20260810 07:52:59.706522 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 34) took 2824 ms (client timeout 1000 ms). Trace:
I20260810 07:52:59.707312 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.708115 1871 tablet_replica.cc:333] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:59.709311 1871 raft_consensus.cc:2243] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:59.709955 1871 raft_consensus.cc:2272] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:52:59.706871 32470 rpcz_store.cc:269] 0810 07:52:56.882554 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:56.882694 (+ 140us) service_pool.cc:224] Handling call
0810 07:52:56.886927 (+ 4233us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:59.706451 (+2819524us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:57.681416 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:57.722358 (+ 40942us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:57.722388 (+ 30us) write_op.cc:435] Acquired schema lock
0810 07:52:57.722400 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:57.722681 (+ 281us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:57.722716 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:57.722743 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:57.722907 (+ 164us) tablet.cc:701] Row locks acquired
0810 07:52:57.722920 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:57.723035 (+ 115us) write_op.cc:270] Start()
0810 07:52:57.723121 (+ 86us) write_op.cc:276] Timestamp: P: 1786348377722982 usec, L: 0
0810 07:52:57.723136 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:52:57.723645 (+ 509us) log.cc:844] Serialized 356 byte log entry
0810 07:52:57.929550 (+205905us) op_driver.cc:464] REPLICATION: finished
0810 07:52:57.934364 (+ 4814us) write_op.cc:301] APPLY: starting
0810 07:52:57.944355 (+ 9991us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:57.985628 (+ 41273us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:57.985740 (+ 112us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:57.985901 (+ 161us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:57.986033 (+ 132us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:57.986084 (+ 51us) write_op.cc:312] APPLY: finished
0810 07:52:57.986247 (+ 163us) log.cc:844] Serialized 23 byte log entry
0810 07:52:57.986948 (+ 701us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.987037 (+ 89us) write_op.cc:454] Released schema lock
0810 07:52:57.987096 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0810 07:52:57.987460 (+ 364us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:57.987509 (+ 49us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":3361408,"child_traces":[["op",{"apply.queue_time_us":1776,"apply.run_cpu_time_us":1442,"apply.run_wall_time_us":53224,"num_ops":1,"prepare.queue_time_us":397724,"prepare.run_cpu_time_us":1963,"prepare.run_wall_time_us":42458,"replication_time_us":206326}]]}
I20260810 07:52:59.714196 860 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:59.715795 1871 ts_tablet_manager.cc:1916] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:59.720338 32460 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 194) took 2049 ms. Trace:
I20260810 07:52:59.720707 32460 rpcz_store.cc:276] 0810 07:52:57.671102 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.813292 (+142190us) service_pool.cc:224] Handling call
0810 07:52:57.933002 (+119710us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c846 00007f30cf429634 00007f30cf4295c2 00007f30cf4294f1 00007f30cf42946c 00007f30cf428722 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:57.933709 (+ 707us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:57.941846 (+ 8137us) catalog_manager.cc:3751] Apply alter schema
0810 07:52:57.943574 (+ 1728us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:52:57.943778 (+ 204us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:52:58.123861 (+180083us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:52:59.720262 (+1596401us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:57.982508 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:57.982921 (+ 413us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:57.982945 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:52:57.982957 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:57.983201 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:57.983235 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:57.983259 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:57.983407 (+ 148us) tablet.cc:701] Row locks acquired
0810 07:52:57.983422 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:52:57.983531 (+ 109us) write_op.cc:270] Start()
0810 07:52:57.983616 (+ 85us) write_op.cc:276] Timestamp: P: 1786348377983456 usec, L: 0
0810 07:52:57.983630 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:52:57.984135 (+ 505us) log.cc:844] Serialized 405 byte log entry
0810 07:52:58.111833 (+127698us) op_driver.cc:464] REPLICATION: finished
0810 07:52:58.116685 (+ 4852us) write_op.cc:301] APPLY: starting
0810 07:52:58.116763 (+ 78us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:58.116858 (+ 95us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:58.116952 (+ 94us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:58.117117 (+ 165us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:58.117247 (+ 130us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:58.122724 (+ 5477us) write_op.cc:312] APPLY: finished
0810 07:52:58.122933 (+ 209us) log.cc:844] Serialized 23 byte log entry
0810 07:52:58.123577 (+ 644us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:58.123633 (+ 56us) write_op.cc:454] Released schema lock
0810 07:52:58.123668 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":15148,"spinlock_wait_cycles":270249728,"child_traces":[["op",{"apply.queue_time_us":4719,"apply.run_cpu_time_us":1325,"apply.run_wall_time_us":7453,"num_ops":1,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":1845,"prepare.run_wall_time_us":2836,"replication_time_us":128124}]]}
I20260810 07:52:59.725217 638 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef (DEFAULT_TABLE table=default.Table_4ec9847b17c44e009c6162f1417d5a13 [id=b70f05a8651b408393dcb8a5cae4507d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.726343 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:59.729027 32462 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fc12205a097c496cba5b777b98f27914 consensus state has no leader
I20260810 07:52:59.729526 866 tablet_service.cc:1511] Processing CreateTablet for tablet 371fa2534984499b95c0945af358e8fc (DEFAULT_TABLE table=default.Table_da3a96ecf800417cbcb7c13b38664a0b [id=d8dcaf8456d64f96aa47eb0192d43201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.730577 1064 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:52:59.731051 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371fa2534984499b95c0945af358e8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.731289 1064 ts_tablet_manager.cc:1403] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.147s user 0.003s sys 0.010s
I20260810 07:52:59.731267 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_da3a96ecf800417cbcb7c13b38664a0b" } modify_external_catalogs: true
I20260810 07:52:59.731770 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_da3a96ecf800417cbcb7c13b38664a0b" } modify_external_catalogs: true
I20260810 07:52:59.734330 860 tablet_service.cc:1511] Processing CreateTablet for tablet 55aaf296555b43f9a27f2926054c8d4b (DEFAULT_TABLE table=default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.735733 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aaf296555b43f9a27f2926054c8d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.739811 1064 raft_consensus.cc:359] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.740583 1064 raft_consensus.cc:385] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.740882 1064 raft_consensus.cc:740] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.741561 1064 consensus_queue.cc:260] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.743633 646 tablet_service.cc:1511] Processing CreateTablet for tablet 55aaf296555b43f9a27f2926054c8d4b (DEFAULT_TABLE table=default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.744735 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aaf296555b43f9a27f2926054c8d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.746452 32460 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d4c51e03eea74f81a39e48e631649307"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.750890 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]) successfully deleted
I20260810 07:52:59.751264 1874 tablet_replica.cc:333] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:52:59.752466 1874 raft_consensus.cc:2243] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:52:59.706403 32467 rpcz_store.cc:276] 0810 07:52:57.340384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.487716 (+147332us) service_pool.cc:224] Handling call
0810 07:52:57.570883 (+ 83167us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c4d6 00007f30cf424b34 00007f30cf424ac2 00007f30cf4249f1 00007f30cf42496c 00007f30cf423c22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:57.572046 (+ 1163us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:52:57.573557 (+ 1511us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:52:57.573690 (+ 133us) catalog_manager.cc:2850] Locking tablets
0810 07:52:57.574421 (+ 731us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:52:58.064239 (+489818us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:52:58.064457 (+ 218us) catalog_manager.cc:2898] Committing in-memory state
0810 07:52:59.704179 (+1639722us) catalog_manager.cc:2902] Aborting table tasks
0810 07:52:59.705947 (+ 1768us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:57.723989 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:57.724377 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:57.724397 (+ 20us) write_op.cc:435] Acquired schema lock
0810 07:52:57.724408 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:57.724696 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:57.771155 (+ 46459us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:57.771184 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:57.771458 (+ 274us) tablet.cc:701] Row locks acquired
0810 07:52:57.771472 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:57.771590 (+ 118us) write_op.cc:270] Start()
0810 07:52:57.771681 (+ 91us) write_op.cc:276] Timestamp: P: 1786348377771531 usec, L: 0
0810 07:52:57.771698 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:52:57.772217 (+ 519us) log.cc:844] Serialized 765 byte log entry
0810 07:52:58.061783 (+289566us) op_driver.cc:464] REPLICATION: finished
0810 07:52:58.062463 (+ 680us) write_op.cc:301] APPLY: starting
0810 07:52:58.062525 (+ 62us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:58.062623 (+ 98us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:58.062718 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:58.062957 (+ 239us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:58.063090 (+ 133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:58.063134 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:52:58.063307 (+ 173us) log.cc:844] Serialized 29 byte log entry
0810 07:52:58.063932 (+ 625us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:58.064000 (+ 68us) write_op.cc:454] Released schema lock
0810 07:52:58.064033 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":185943424,"child_traces":[["op",{"apply.queue_time_us":512,"apply.run_cpu_time_us":1361,"apply.run_wall_time_us":2163,"num_ops":2,"prepare.queue_time_us":147485,"prepare.run_cpu_time_us":2057,"prepare.run_wall_time_us":48422,"replication_time_us":289984}]]}
I20260810 07:52:59.766983 32467 master_service.cc:960] ReplaceTablet: received request to replace tablet 08520a2ac0f64c3abd534a32100839a4 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:59.772631 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a9ab67614949bb93925144c4675f01 with delete_type TABLET_DATA_DELETED (replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:59.781868 627 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.782291 1047 tablet_bootstrap.cc:654] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:52:59.783689 627 tablet_service.cc:1511] Processing CreateTablet for tablet 371fa2534984499b95c0945af358e8fc (DEFAULT_TABLE table=default.Table_da3a96ecf800417cbcb7c13b38664a0b [id=d8dcaf8456d64f96aa47eb0192d43201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.784848 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.788807 32469 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f] alter complete (version 1)
W20260810 07:52:59.795588 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 22) took 2050 ms (client timeout 1000 ms). Trace:
W20260810 07:52:59.795922 32465 rpcz_store.cc:269] 0810 07:52:57.745564 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.771651 (+ 26087us) service_pool.cc:224] Handling call
0810 07:52:59.795519 (+2023868us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19766,"reported_tablets":17,"spinlock_wait_cycles":875904}
I20260810 07:52:59.804091 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371fa2534984499b95c0945af358e8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.811265 645 tablet_service.cc:1511] Processing CreateTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.812288 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a97561b5ff5490890e756ca5ff50d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.815974 639 tablet_service.cc:1511] Processing CreateTablet for tablet 01d24209ef6b4c919ce12a702a318577 (DEFAULT_TABLE table=default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.817175 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d24209ef6b4c919ce12a702a318577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.822489 1064 ts_tablet_manager.cc:1434] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.091s user 0.008s sys 0.000s
I20260810 07:52:59.824851 872 tablet_service.cc:1511] Processing CreateTablet for tablet d3077b3fe04e406a89560229b76d8242 (DEFAULT_TABLE table=default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.825486 646 tablet_service.cc:1511] Processing CreateTablet for tablet d3077b3fe04e406a89560229b76d8242 (DEFAULT_TABLE table=default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.826526 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3077b3fe04e406a89560229b76d8242. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:52:59.777230 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 35) took 1878 ms (client timeout 1000 ms). Trace:
W20260810 07:52:59.834251 32463 rpcz_store.cc:269] 0810 07:52:57.898534 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.898669 (+ 135us) service_pool.cc:224] Handling call
0810 07:52:59.777153 (+1878484us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":12}
I20260810 07:52:59.835273 32463 master_service.cc:960] ReplaceTablet: received request to replace tablet 2c171d3358e44ae2962ab3c4c377b0f1 from {username='slave'} at 127.0.0.1:43806
I20260810 07:52:59.837543 1064 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:52:59.748482 32460 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4c51e03eea74f81a39e48e631649307 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.845037 869 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:55008
I20260810 07:52:59.824200 1874 raft_consensus.cc:2272] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:59.853601 377 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe1a943230432dbe533e7b3671172b (DEFAULT_TABLE table=default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.854240 32467 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 08520a2ac0f64c3abd534a32100839a4
I20260810 07:52:59.854739 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe1a943230432dbe533e7b3671172b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.855721 32467 catalog_manager.cc:6807] ReplaceTablet: tablet 08520a2ac0f64c3abd534a32100839a4 deleted and replaced by tablet a61dbafb4f7444e690cea0d003d38f22
W20260810 07:52:59.779372 32461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 25) took 1178 ms (client timeout 1000 ms). Trace:
W20260810 07:52:59.857569 32461 rpcz_store.cc:269] 0810 07:52:58.601224 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:58.712236 (+111012us) service_pool.cc:224] Handling call
0810 07:52:59.779304 (+1067068us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:59.395004 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:59.395410 (+ 406us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:59.395434 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:52:59.395446 (+ 12us) tablet.cc:662] Decoding operations
0810 07:52:59.395734 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:59.395768 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:59.395791 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:52:59.396060 (+ 269us) tablet.cc:701] Row locks acquired
0810 07:52:59.396074 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:52:59.396201 (+ 127us) write_op.cc:270] Start()
0810 07:52:59.396280 (+ 79us) write_op.cc:276] Timestamp: P: 1786348379396129 usec, L: 0
0810 07:52:59.396294 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:52:59.396843 (+ 549us) log.cc:844] Serialized 913 byte log entry
0810 07:52:59.451965 (+ 55122us) op_driver.cc:464] REPLICATION: finished
0810 07:52:59.454251 (+ 2286us) write_op.cc:301] APPLY: starting
0810 07:52:59.454311 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:59.454414 (+ 103us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:59.454511 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:59.454777 (+ 266us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:59.454913 (+ 136us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:52:59.454972 (+ 59us) write_op.cc:312] APPLY: finished
0810 07:52:59.455154 (+ 182us) log.cc:844] Serialized 29 byte log entry
0810 07:52:59.455307 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:59.455374 (+ 67us) write_op.cc:454] Released schema lock
0810 07:52:59.455412 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":53978,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":2124,"apply.run_cpu_time_us":1461,"apply.run_wall_time_us":1461,"num_ops":2,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":6172,"prepare.run_wall_time_us":34108,"raft.queue_time_us":26908,"raft.run_cpu_time_us":25,"raft.run_wall_time_us":26,"replication_time_us":55567,"thread_start_us":31783,"threads_started":1}]]}
I20260810 07:52:59.861502 625 tablet_service.cc:1511] Processing CreateTablet for tablet a3642f29b38241be9db3cf37ccde2039 (DEFAULT_TABLE table=default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.862154 869 tablet_service.cc:1511] Processing CreateTablet for tablet 5d7024a39aa040c2b00196f668b776e8 (DEFAULT_TABLE table=default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.862785 625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3642f29b38241be9db3cf37ccde2039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.863147 869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d7024a39aa040c2b00196f668b776e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.864403 855 tablet_service.cc:1511] Processing CreateTablet for tablet 01d24209ef6b4c919ce12a702a318577 (DEFAULT_TABLE table=default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.865671 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d24209ef6b4c919ce12a702a318577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.873561 644 tablet_service.cc:1511] Processing CreateTablet for tablet 5d7024a39aa040c2b00196f668b776e8 (DEFAULT_TABLE table=default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.874943 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d7024a39aa040c2b00196f668b776e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.878921 642 tablet_service.cc:1511] Processing CreateTablet for tablet 834a48ebb87a4b999722e05af1feab21 (DEFAULT_TABLE table=default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.881443 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834a48ebb87a4b999722e05af1feab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.881858 1871 ts_tablet_manager.cc:1929] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:52:59.890125 1871 log.cc:1199] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/9ae4a28e89714c65b7551f36d1dc78b4
I20260810 07:52:59.893184 852 tablet_service.cc:1511] Processing CreateTablet for tablet 834a48ebb87a4b999722e05af1feab21 (DEFAULT_TABLE table=default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.894670 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834a48ebb87a4b999722e05af1feab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.877207 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c26400e44124423eb9b7db04cf875aba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:52:59.904907 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c26400e44124423eb9b7db04cf875aba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:52:59.908280 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f3d034925e2e4dd6b7b0afff1c67e87d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.911547 1871 ts_tablet_manager.cc:1950] T 9ae4a28e89714c65b7551f36d1dc78b4 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:59.856063 376 tablet_service.cc:1511] Processing CreateTablet for tablet 98dec9b072ac49959181003cb78567af (DEFAULT_TABLE table=default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.908016 382 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef (DEFAULT_TABLE table=default.Table_4ec9847b17c44e009c6162f1417d5a13 [id=b70f05a8651b408393dcb8a5cae4507d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.913558 384 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e0a84839427099a9da06e2b970ce (DEFAULT_TABLE table=default.Table_8bed0806196e4e6393b129851ab29404 [id=a4439e8baa3e43d5980744518075a788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.913949 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.914489 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e0a84839427099a9da06e2b970ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.914597 1871 tablet_replica.cc:333] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:59.915709 1871 raft_consensus.cc:2243] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:59.916332 1871 raft_consensus.cc:2272] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:52:59.897351 1552 raft_consensus.cc:493] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:52:59.917722 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98dec9b072ac49959181003cb78567af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:52:59.917268 1552 raft_consensus.cc:515] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.918655 1047 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:52:59.919066 1047 ts_tablet_manager.cc:1403] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.442s user 0.013s sys 0.000s
I20260810 07:52:59.925925 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 9ae4a28e89714c65b7551f36d1dc78b4 (table default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]) successfully deleted
I20260810 07:52:59.926311 1552 leader_election.cc:290] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:52:59.927243 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.927789 1871 ts_tablet_manager.cc:1916] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:52:59.929916 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.950552 1047 raft_consensus.cc:359] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.951318 1047 raft_consensus.cc:385] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.951596 1047 raft_consensus.cc:740] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.903445 631 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a9ab67614949bb93925144c4675f01 with delete_type TABLET_DATA_DELETED (replaced by tablet e2fe3153512b40a99caf3fb9b81fd7a1) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.902633 624 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.952615 1047 consensus_queue.cc:260] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.978895 1047 ts_tablet_manager.cc:1434] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.060s user 0.006s sys 0.002s
I20260810 07:52:59.979873 1047 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:52:59.987565 1871 ts_tablet_manager.cc:1929] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:52:59.988092 1871 log.cc:1199] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:52:59.988938 1871 ts_tablet_manager.cc:1950] T cf45a33485e9463d89b5ce8affa0bd22 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:52:59.988950 1071 raft_consensus.cc:359] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.989619 1071 raft_consensus.cc:385] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:52:59.989880 1071 raft_consensus.cc:740] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:52:59.990549 1071 consensus_queue.cc:260] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:52:59.992298 1871 tablet_replica.cc:333] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:52:59.911536 1880 ts_tablet_manager.cc:1929] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:52:59.997129 1880 log.cc:1199] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/20a9ab67614949bb93925144c4675f01
I20260810 07:52:59.997915 1871 raft_consensus.cc:2243] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:52:59.998350 1880 ts_tablet_manager.cc:1950] T 20a9ab67614949bb93925144c4675f01 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:52:59.998495 1871 raft_consensus.cc:2272] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:00.001240 1871 ts_tablet_manager.cc:1916] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:00.009455 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.009842 371 tablet_service.cc:1558] Processing DeleteTablet for tablet 4af5109e338844139444bb7111619195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:49 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.010985 371 tablet_service.cc:1558] Processing DeleteTablet for tablet c7c136422ffe4bb2b4f77ab081acd19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.012655 371 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.012913 377 tablet_service.cc:1511] Processing CreateTablet for tablet 55aaf296555b43f9a27f2926054c8d4b (DEFAULT_TABLE table=default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.014156 1880 tablet_replica.cc:333] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:00.015468 1880 raft_consensus.cc:2243] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:00.021201 1880 raft_consensus.cc:2272] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:00.022033 385 tablet_service.cc:1511] Processing CreateTablet for tablet 371fa2534984499b95c0945af358e8fc (DEFAULT_TABLE table=default.Table_da3a96ecf800417cbcb7c13b38664a0b [id=d8dcaf8456d64f96aa47eb0192d43201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.024127 1880 ts_tablet_manager.cc:1916] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:00.024221 371 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:52:59.992584 1071 ts_tablet_manager.cc:1434] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.306s user 0.003s sys 0.001s
I20260810 07:53:00.027261 1047 tablet_bootstrap.cc:654] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:00.030829 1071 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:00.037468 1071 tablet_bootstrap.cc:654] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:00.053705 1552 raft_consensus.cc:493] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.054402 1552 raft_consensus.cc:515] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.056596 1552 leader_election.cc:290] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.061913 371 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.026343 859 tablet_service.cc:1511] Processing CreateTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.063881 371 tablet_service.cc:1511] Processing CreateTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:52:59.909965 851 tablet_service.cc:1511] Processing CreateTablet for tablet a3642f29b38241be9db3cf37ccde2039 (DEFAULT_TABLE table=default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.066712 1047 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:00.067328 1047 ts_tablet_manager.cc:1403] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.088s user 0.011s sys 0.002s
I20260810 07:53:00.070482 1047 raft_consensus.cc:359] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.070065 1753 raft_consensus.cc:493] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.071139 1047 raft_consensus.cc:385] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:00.082677 1753 raft_consensus.cc:515] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.083127 1047 raft_consensus.cc:740] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:00.083791 1047 consensus_queue.cc:260] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.084686 1753 leader_election.cc:290] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:00.087476 1913 raft_consensus.cc:493] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.087975 1913 raft_consensus.cc:515] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.089911 1913 leader_election.cc:290] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:00.095455 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a97561b5ff5490890e756ca5ff50d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.070550 849 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:00.102497 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3077b3fe04e406a89560229b76d8242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.105647 1874 ts_tablet_manager.cc:1916] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:00.107350 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3642f29b38241be9db3cf37ccde2039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.107903 380 tablet_service.cc:1511] Processing CreateTablet for tablet 01d24209ef6b4c919ce12a702a318577 (DEFAULT_TABLE table=default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.109897 369 tablet_service.cc:1511] Processing CreateTablet for tablet 834a48ebb87a4b999722e05af1feab21 (DEFAULT_TABLE table=default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.112090 368 tablet_service.cc:1511] Processing CreateTablet for tablet 5d7024a39aa040c2b00196f668b776e8 (DEFAULT_TABLE table=default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.114270 367 tablet_service.cc:1511] Processing CreateTablet for tablet a3642f29b38241be9db3cf37ccde2039 (DEFAULT_TABLE table=default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.115497 364 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:00.121464 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 195536us
I20260810 07:53:00.123440 627 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.902259 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 371fa2534984499b95c0945af358e8fc
I20260810 07:53:00.126261 638 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:57706
I20260810 07:52:59.906865 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_9d5653ef6b2141efa61961b2fdbed04c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:52:59.967765 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_0f57e2f93400402d8c7458cf70c65ccc" } new_table_name: "default.Table_f2a1788b7d16410e92f4494c0bfaa865" modify_external_catalogs: true
W20260810 07:53:00.138324 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d5653ef6b2141efa61961b2fdbed04c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:00.060992 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 24) took 2529 ms (client timeout 1000 ms). Trace:
W20260810 07:53:00.142099 32464 rpcz_store.cc:269] 0810 07:52:57.531958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:57.610923 (+ 78965us) service_pool.cc:224] Handling call
0810 07:53:00.060903 (+2449980us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":19}
I20260810 07:53:00.144097 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3523dbadfd954f2da7cd8e456fffa4c4" } modify_external_catalogs: true
I20260810 07:53:00.144979 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3523dbadfd954f2da7cd8e456fffa4c4" } modify_external_catalogs: true
I20260810 07:53:00.145691 849 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:00.147617 792 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:00.148051 792 leader_election.cc:304] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:00.101876 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3d034925e2e4dd6b7b0afff1c67e87d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.146800 849 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:00.151232 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 371fa2534984499b95c0945af358e8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:59 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:00.153576 1064 tablet_bootstrap.cc:654] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:00.163996 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 has no consensus state
W20260810 07:53:00.113407 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_f7f1ce7f0e8848ffa1dc9250b95fa516 [id=f26d526a9f1949df98cb9ff907fbfe73], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fc12205a097c496cba5b777b98f27914 consensus state has no leader
I20260810 07:53:00.128458 32462 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_5c9348ac7cc14fb1842cb6bd08652cae" } new_table_name: "default.Table_d8cf507b1d004b598ab0cb354102eec1" modify_external_catalogs: true
I20260810 07:53:00.186846 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:00.192174 1047 ts_tablet_manager.cc:1434] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.125s user 0.005s sys 0.001s
I20260810 07:53:00.192710 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:00.193348 678 raft_consensus.cc:2468] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:00.197309 1047 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:00.208447 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:53:00.209071 1916 raft_consensus.cc:2749] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:00.169340 373 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:00.213068 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 23) took 1506 ms (client timeout 1000 ms). Trace:
I20260810 07:53:00.214210 1871 ts_tablet_manager.cc:1929] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:00.214834 1871 log.cc:1199] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:00.215775 1871 ts_tablet_manager.cc:1950] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:00.181924 375 tablet_service.cc:1511] Processing CreateTablet for tablet d3077b3fe04e406a89560229b76d8242 (DEFAULT_TABLE table=default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:00.169104 1821 raft_consensus.cc:493] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.216490 1821 raft_consensus.cc:515] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.220096 1821 leader_election.cc:290] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.220796 1821 raft_consensus.cc:493] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.221529 1821 raft_consensus.cc:515] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.222967 1600 raft_consensus.cc:493] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:53:00.223474 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 99479.1us
I20260810 07:53:00.223585 1600 raft_consensus.cc:515] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.196696 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55aaf296555b43f9a27f2926054c8d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.201512 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:00.206350 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:00.225839 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet cf45a33485e9463d89b5ce8affa0bd22 (table default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]) successfully deleted
I20260810 07:53:00.229177 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a97561b5ff5490890e756ca5ff50d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.231225 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d24209ef6b4c919ce12a702a318577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.234342 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371fa2534984499b95c0945af358e8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.237001 368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d7024a39aa040c2b00196f668b776e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.238168 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3642f29b38241be9db3cf37ccde2039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.239171 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834a48ebb87a4b999722e05af1feab21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:00.239313 638 tablet_service.cc:1558] Processing DeleteTablet for tablet 371fa2534984499b95c0945af358e8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:59 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:00.241276 1600 leader_election.cc:290] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.224704 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:00.251884 676 raft_consensus.cc:2468] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:00.253309 308 leader_election.cc:304] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:00.254698 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3077b3fe04e406a89560229b76d8242. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:00.254612 308 leader_election.cc:343] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:00.263156 1930 raft_consensus.cc:2804] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:00.263720 1930 raft_consensus.cc:493] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:00.264114 1930 raft_consensus.cc:3060] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.271852 1871 tablet_replica.cc:333] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:00.273027 1871 raft_consensus.cc:2243] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:00.274719 1871 raft_consensus.cc:2272] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:00.279091 1071 tablet_bootstrap.cc:492] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:00.279548 1071 ts_tablet_manager.cc:1403] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.249s user 0.012s sys 0.012s
I20260810 07:53:00.285405 1047 tablet_bootstrap.cc:654] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:00.288535 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:00.297540 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:00.304550 792 consensus_peers.cc:597] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:00.305508 574 leader_election.cc:343] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:00.305138 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f50ce5293cd74b41a97f9a6b66c60211"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:00.307262 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f50ce5293cd74b41a97f9a6b66c60211 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.289256 891 raft_consensus.cc:2468] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:53:00.315996 792 leader_election.cc:343] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:00.316044 1071 raft_consensus.cc:359] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.316782 1071 raft_consensus.cc:385] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:00.317058 1071 raft_consensus.cc:740] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:00.317524 792 leader_election.cc:343] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:00.317692 1071 consensus_queue.cc:260] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.322217 1871 ts_tablet_manager.cc:1916] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:00.332938 792 leader_election.cc:343] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:00.333472 792 leader_election.cc:304] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:00.213457 32469 rpcz_store.cc:269] 0810 07:52:58.706614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:58.982007 (+275393us) service_pool.cc:224] Handling call
0810 07:52:59.157381 (+175374us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:52:59.757734 (+600353us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:59.757762 (+ 28us) write_op.cc:454] Released schema lock
0810 07:53:00.212986 (+455224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:59.736127 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:52:59.736515 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:52:59.736540 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:52:59.736551 (+ 11us) tablet.cc:662] Decoding operations
0810 07:52:59.736841 (+ 290us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:52:59.737010 (+ 169us) write_op.cc:641] Partition lock acquired for write op
0810 07:52:59.737036 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:52:59.737196 (+ 160us) tablet.cc:701] Row locks acquired
0810 07:52:59.737209 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:52:59.737332 (+ 123us) write_op.cc:270] Start()
0810 07:52:59.737421 (+ 89us) write_op.cc:276] Timestamp: P: 1786348379737282 usec, L: 0
0810 07:52:59.737437 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:52:59.738009 (+ 572us) log.cc:844] Serialized 356 byte log entry
0810 07:52:59.747265 (+ 9256us) op_driver.cc:464] REPLICATION: finished
0810 07:52:59.747786 (+ 521us) write_op.cc:301] APPLY: starting
0810 07:52:59.747904 (+ 118us) tablet.cc:1367] starting BulkCheckPresence
0810 07:52:59.748017 (+ 113us) tablet.cc:1370] finished BulkCheckPresence
0810 07:52:59.748260 (+ 243us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:52:59.748447 (+ 187us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:52:59.748696 (+ 249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:52:59.748775 (+ 79us) write_op.cc:312] APPLY: finished
0810 07:52:59.749059 (+ 284us) log.cc:844] Serialized 23 byte log entry
0810 07:52:59.749280 (+ 221us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:52:59.749358 (+ 78us) write_op.cc:454] Released schema lock
0810 07:52:59.749404 (+ 46us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":22,"spinlock_wait_cycles":369601920,"child_traces":[["op",{"apply.queue_time_us":221,"apply.run_cpu_time_us":1896,"apply.run_wall_time_us":1894,"num_ops":1,"prepare.queue_time_us":35299,"prepare.run_cpu_time_us":2255,"prepare.run_wall_time_us":2256,"raft.queue_time_us":1249,"raft.run_cpu_time_us":1332,"raft.run_wall_time_us":2503,"replication_time_us":9713}]]}
I20260810 07:53:00.339259 32469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_aa23aa2f7ea64c1387df86e35a66e368" } modify_external_catalogs: true
W20260810 07:53:00.341101 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 36) took 1270 ms (client timeout 1000 ms). Trace:
I20260810 07:53:00.346992 1047 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:00.347469 1047 ts_tablet_manager.cc:1403] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.150s user 0.008s sys 0.003s
I20260810 07:53:00.350244 1047 raft_consensus.cc:359] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.350914 1047 raft_consensus.cc:385] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:00.351187 1047 raft_consensus.cc:740] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:00.375531 1047 consensus_queue.cc:260] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:00.382128 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 49223.3us
I20260810 07:53:00.381265 1926 consensus_queue.cc:1048] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260810 07:53:00.383077 1916 consensus_queue.cc:1048] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:00.390753 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:00.391597 890 raft_consensus.cc:2468] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:53:00.353442 32462 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 213fa4838fe847de87091cfba97eaedc consensus state has no leader
I20260810 07:53:00.359246 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 4 got 3
W20260810 07:53:00.395651 32468 rpcz_store.cc:269] 0810 07:52:59.070637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:59.213388 (+142751us) service_pool.cc:224] Handling call
0810 07:52:59.251155 (+ 37767us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:00.341013 (+1089858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92136,"reported_tablets":21,"spinlock_wait_cycles":70780672}
I20260810 07:53:00.396123 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:00.396942 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_8880a03671844a10931425b1c422e0da" } new_table_name: "default.Table_1a8cb51949d74f3491dbc79f5d488eb6" modify_external_catalogs: true
I20260810 07:53:00.396759 32470 catalog_manager.cc:5719] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:00.397970 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:00.398717 32470 catalog_manager.cc:5719] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:00.421964 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:00.421908 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "449359c8e9fa4756ba149f3dce6e0ce7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:00.423125 413 raft_consensus.cc:2468] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:00.223794 1821 leader_election.cc:290] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.399338 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3cec5176ba3946aebc2391756561ecc1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:00.441520 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cec5176ba3946aebc2391756561ecc1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.434022 32469 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_aa23aa2f7ea64c1387df86e35a66e368" } modify_external_catalogs: true
I20260810 07:53:00.442977 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:00.445672 1047 ts_tablet_manager.cc:1434] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.098s user 0.007s sys 0.001s
W20260810 07:53:00.448902 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 111865us
I20260810 07:53:00.449596 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]) successfully deleted
I20260810 07:53:00.451308 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 449359c8e9fa4756ba149f3dce6e0ce7 (table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]) successfully deleted
W20260810 07:53:00.451959 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because the tablet was not found. No further retry: Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:53:00.389560 1913 raft_consensus.cc:2749] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:00.474184 1931 log.cc:927] Time spent T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260810 07:53:00.482245 1047 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:00.487833 1047 tablet_bootstrap.cc:654] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:00.503063 1871 ts_tablet_manager.cc:1929] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260810 07:53:00.508607 1874 ts_tablet_manager.cc:1929] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:00.514526 1064 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:00.515334 1064 ts_tablet_manager.cc:1403] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.678s user 0.010s sys 0.015s
I20260810 07:53:00.517619 1871 log.cc:1199] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/08520a2ac0f64c3abd534a32100839a4
I20260810 07:53:00.522207 1871 ts_tablet_manager.cc:1950] T 08520a2ac0f64c3abd534a32100839a4 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:00.523379 862 consensus_meta.cc:287] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.623s user 0.012s sys 0.000s
W20260810 07:53:00.526844 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 144011us
I20260810 07:53:00.527585 792 leader_election.cc:304] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:00.528563 1916 raft_consensus.cc:2804] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:00.528915 1916 raft_consensus.cc:493] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:00.529201 1916 raft_consensus.cc:3060] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.541040 1916 raft_consensus.cc:515] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.545310 574 leader_election.cc:304] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:00.546198 1600 raft_consensus.cc:2804] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:00.546527 1600 raft_consensus.cc:493] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:00.546811 1600 raft_consensus.cc:3060] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.553712 1600 raft_consensus.cc:515] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:00.555527 309 leader_election.cc:343] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:00.555995 309 leader_election.cc:304] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:00.557392 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 104765us
I20260810 07:53:00.557590 1552 consensus_queue.cc:1048] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:00.568439 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 73574.4us
I20260810 07:53:00.572544 1874 log.cc:1199] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/9ae4a28e89714c65b7551f36d1dc78b4
I20260810 07:53:00.575327 574 leader_election.cc:304] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:00.576236 1552 raft_consensus.cc:2804] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:00.576571 1552 raft_consensus.cc:493] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:00.576933 1552 raft_consensus.cc:3060] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.591197 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:00.593207 1600 leader_election.cc:290] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.600523 1821 raft_consensus.cc:493] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:00.601106 1821 raft_consensus.cc:515] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.574110 1064 raft_consensus.cc:359] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.603133 1821 leader_election.cc:290] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.603819 1821 raft_consensus.cc:2749] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:00.611500 1064 raft_consensus.cc:385] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:00.612049 1064 raft_consensus.cc:740] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:00.613278 1064 consensus_queue.cc:260] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.613309 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_eb0339f23b7749ccb8c8081ee9dcae9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:00.615010 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb0339f23b7749ccb8c8081ee9dcae9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.614964 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:00.617128 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a" } new_table_name: "default.Table_170c952671b84b36873085f10e575663" modify_external_catalogs: true
I20260810 07:53:00.625240 380 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.626410 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 371fa2534984499b95c0945af358e8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:59 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.638507 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.638355 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_cedb584fa8d744b5ae31de6b73b34c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:00.641201 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0539c0c6e0054726834e1dfe114e4ef7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:00.650112 1916 leader_election.cc:290] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:00.651044 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0539c0c6e0054726834e1dfe114e4ef7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.610770 1874 ts_tablet_manager.cc:1950] T 9ae4a28e89714c65b7551f36d1dc78b4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
W20260810 07:53:00.654722 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cedb584fa8d744b5ae31de6b73b34c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.656625 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 9ae4a28e89714c65b7551f36d1dc78b4 (table default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]) successfully deleted
I20260810 07:53:00.660373 1880 ts_tablet_manager.cc:1929] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:00.665912 1552 raft_consensus.cc:515] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:00.669109 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 113623us
I20260810 07:53:00.679451 1930 raft_consensus.cc:515] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.673666 1880 log.cc:1199] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/9ae4a28e89714c65b7551f36d1dc78b4
W20260810 07:53:00.679912 309 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:53:00.690740 411 raft_consensus.cc:2393] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:00.697921 1064 ts_tablet_manager.cc:1434] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.182s user 0.005s sys 0.002s
I20260810 07:53:00.704056 377 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:00.704955 1880 ts_tablet_manager.cc:1950] T 9ae4a28e89714c65b7551f36d1dc78b4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:00.710193 1880 tablet_replica.cc:333] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:00.710455 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:00.711179 894 raft_consensus.cc:3060] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.711558 1930 leader_election.cc:290] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.713240 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:00.717805 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 9ae4a28e89714c65b7551f36d1dc78b4 (table default.Table_3d59e31692ec40109b7e69c59019dd83 [id=9effee1773854239b6bcb25316754be1]) successfully deleted
W20260810 07:53:00.718866 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 4af5109e338844139444bb7111619195 because the tablet was not found. No further retry: Not found: Tablet not found: 4af5109e338844139444bb7111619195
W20260810 07:53:00.719944 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet c7c136422ffe4bb2b4f77ab081acd19f because the tablet was not found. No further retry: Not found: Tablet not found: c7c136422ffe4bb2b4f77ab081acd19f
I20260810 07:53:00.728906 1930 raft_consensus.cc:493] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:53:00.729022 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 20a9ab67614949bb93925144c4675f01 because the tablet was not found. No further retry: Not found: Tablet not found: 20a9ab67614949bb93925144c4675f01
I20260810 07:53:00.750003 1064 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:00.771080 1047 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:00.771709 1047 ts_tablet_manager.cc:1403] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.290s user 0.007s sys 0.005s
I20260810 07:53:00.775234 1047 raft_consensus.cc:359] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:00.782608 792 consensus_peers.cc:597] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf45a33485e9463d89b5ce8affa0bd22. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:00.791942 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:00.796926 1071 ts_tablet_manager.cc:1434] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.517s user 0.016s sys 0.012s
I20260810 07:53:00.799651 1968 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260810 07:53:00.802106 792 leader_election.cc:343] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:00.797323 1071 ts_tablet_manager.cc:1467] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Tablet startup took 517ms
W20260810 07:53:00.817705 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8718f998623e4f1c8b7b8be815e184a3 consensus state has no leader
I20260810 07:53:00.818992 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_6ff328bbdb89429a8b773fbdc529c27c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:00.820163 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 09712cfebab34e56ba844e7aa0e8d152 because the tablet was not found. No further retry: Not found: Tablet not found: 09712cfebab34e56ba844e7aa0e8d152
W20260810 07:53:00.821559 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ff328bbdb89429a8b773fbdc529c27c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:00.816907 1047 raft_consensus.cc:385] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:00.829787 1047 raft_consensus.cc:740] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:00.830917 1047 consensus_queue.cc:260] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.831483 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]) successfully deleted
I20260810 07:53:00.833077 1047 ts_tablet_manager.cc:1434] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.061s user 0.002s sys 0.005s
W20260810 07:53:00.835464 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet c7c136422ffe4bb2b4f77ab081acd19f because the tablet was not found. No further retry: Not found: Tablet not found: c7c136422ffe4bb2b4f77ab081acd19f
W20260810 07:53:00.836410 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 20a9ab67614949bb93925144c4675f01 because the tablet was not found. No further retry: Not found: Tablet not found: 20a9ab67614949bb93925144c4675f01
I20260810 07:53:00.790612 1930 raft_consensus.cc:515] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:00.839594 1930 leader_election.cc:290] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.840595 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:00.841265 893 raft_consensus.cc:3060] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:00.842207 1970 tablet.cc:1724] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:00.844251 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:00.844529 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 09712cfebab34e56ba844e7aa0e8d152 because the tablet was not found. No further retry: Not found: Tablet not found: 09712cfebab34e56ba844e7aa0e8d152
W20260810 07:53:00.845494 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet b2bfffdd01524eada0a85c1a525325a0 because the tablet was not found. No further retry: Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0
I20260810 07:53:00.838685 1874 tablet_replica.cc:333] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
W20260810 07:53:00.846964 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 371fa2534984499b95c0945af358e8fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 371fa2534984499b95c0945af358e8fc already in progress: creating tablet
W20260810 07:53:00.850580 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 87180.6us
I20260810 07:53:00.770278 1552 leader_election.cc:290] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:00.778468 1963 consensus_queue.cc:1048] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:00.782562 1956 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260810 07:53:00.785514 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 169819us
I20260810 07:53:00.785838 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:00.856521 678 raft_consensus.cc:3060] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:00.824872 1071 ts_tablet_manager.cc:1469] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Trace:
0810 07:52:54.087301 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:54.089054 (+ 1753us) service_pool.cc:224] Handling call
0810 07:52:54.090468 (+ 1414us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:54.090553 (+ 85us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:54.110856 (+ 20303us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:54.203697 (+ 92841us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:54.204151 (+ 454us) inbound_call.cc:177] Queueing success response
0810 07:53:00.030277 (+5826126us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0810 07:53:00.279794 (+249517us) ts_tablet_manager.cc:1435] Starting tablet replica
0810 07:53:00.280275 (+ 481us) tablet_replica.cc:261] Starting instrumentation
0810 07:53:00.280774 (+ 499us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":253200,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":22983,"tablet-open.queue_time_us":5826240,"thread_start_us":430598,"threads_started":1}
I20260810 07:53:00.836331 1880 raft_consensus.cc:2243] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:00.868794 1874 raft_consensus.cc:2243] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:00.869702 1064 tablet_bootstrap.cc:654] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:00.869638 1874 raft_consensus.cc:2272] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:00.871341 894 raft_consensus.cc:2468] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:00.871903 1874 ts_tablet_manager.cc:1916] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:00.873574 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6226a2377864d47b14901df39677127" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:00.869140 1880 raft_consensus.cc:2272] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:00.882041 893 raft_consensus.cc:2468] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:00.879170 1071 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:00.889477 1966 tablet.cc:1724] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:00.959295 1880 ts_tablet_manager.cc:1916] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:00.971591 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 77183.6us
I20260810 07:53:00.972546 32465 master_service.cc:960] ReplaceTablet: received request to replace tablet 45c5e309389d402092f59c801923854a from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:00.987918 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc89d15b4d544b7bc8abc12a4c6d3e5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:00.990465 308 leader_election.cc:343] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:53:00.991087 308 leader_election.cc:304] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:00.992249 308 raft_consensus.cc:2686] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260810 07:53:01.003636 677 raft_consensus.cc:2393] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:00.851117 1047 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:00.776024 1948 consensus_queue.cc:1048] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:01.016467 1064 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:01.017127 1064 ts_tablet_manager.cc:1403] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.318s user 0.020s sys 0.010s
I20260810 07:53:01.020037 1064 raft_consensus.cc:359] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.021785 308 leader_election.cc:304] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:01.022912 1946 raft_consensus.cc:2749] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:01.022998 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f7f1ce7f0e8848ffa1dc9250b95fa516" } new_table_name: "default.Table_e972e7a884334e7aaf450c0be103a827" modify_external_catalogs: true
W20260810 07:53:01.032011 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 167997us
I20260810 07:53:01.035660 32462 master_service.cc:960] ReplaceTablet: received request to replace tablet 0098aaa7be644c9885167dc1ee4929b5 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:01.040766 1064 raft_consensus.cc:385] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:01.041154 1064 raft_consensus.cc:740] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:01.041172 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 68924.8us
I20260810 07:53:01.042035 574 leader_election.cc:304] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:01.041949 1064 consensus_queue.cc:260] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:01.055603 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fc12205a097c496cba5b777b98f27914 consensus state has no leader
I20260810 07:53:01.058480 1071 tablet_bootstrap.cc:654] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.091584 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_7438e01a7d4a499c856ce0f80c8666c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:01.093158 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1faed468bfa4d1ba5d92c0b9b194781" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:01.096221 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7438e01a7d4a499c856ce0f80c8666c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.102873 1064 ts_tablet_manager.cc:1434] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.085s user 0.006s sys 0.001s
W20260810 07:53:01.106305 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 64367.4us
I20260810 07:53:01.106304 308 leader_election.cc:304] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:01.107126 574 leader_election.cc:304] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:01.107681 1946 raft_consensus.cc:2749] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:01.107998 1064 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:01.110193 642 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (DEFAULT_TABLE table=default.Table_749f194454fe43a88634da1983fc5584 [id=13a2b62a847e491ebd5208c432704017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.111347 642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6c33f86da645e9bf50e45bc229d0a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.119248 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_f2a1788b7d16410e92f4494c0bfaa865 [id=7e8c44d3660746d69e0982f1cf2dc80e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 consensus state has no leader
I20260810 07:53:01.121692 375 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (DEFAULT_TABLE table=default.Table_749f194454fe43a88634da1983fc5584 [id=13a2b62a847e491ebd5208c432704017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.130862 1971 raft_consensus.cc:2804] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:01.135689 862 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (DEFAULT_TABLE table=default.Table_749f194454fe43a88634da1983fc5584 [id=13a2b62a847e491ebd5208c432704017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.137655 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6c33f86da645e9bf50e45bc229d0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.138820 1987 raft_consensus.cc:2804] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:01.140231 644 tablet_service.cc:1511] Processing CreateTablet for tablet 999f3e79ee4b4c368b60b202735c2556 (DEFAULT_TABLE table=default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.141369 644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999f3e79ee4b4c368b60b202735c2556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.142072 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d83c2642cc224dc69bb060597fbdfc63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:01.143051 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6c33f86da645e9bf50e45bc229d0a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.143648 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d83c2642cc224dc69bb060597fbdfc63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.145285 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 0098aaa7be644c9885167dc1ee4929b5
I20260810 07:53:01.147387 32462 catalog_manager.cc:6807] ReplaceTablet: tablet 0098aaa7be644c9885167dc1ee4929b5 deleted and replaced by tablet e1fa9200140a41dfbd886442ae336988
I20260810 07:53:01.149212 376 tablet_service.cc:1511] Processing CreateTablet for tablet 999f3e79ee4b4c368b60b202735c2556 (DEFAULT_TABLE table=default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.149559 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:01.150309 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999f3e79ee4b4c368b60b202735c2556. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.153119 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0098aaa7be644c9885167dc1ee4929b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0098aaa7be644c9885167dc1ee4929b5 already in progress: creating tablet
I20260810 07:53:01.155709 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:01.157927 368 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:01.160686 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 45c5e309389d402092f59c801923854a
I20260810 07:53:01.152645 1971 raft_consensus.cc:697] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:01.163193 32465 catalog_manager.cc:6807] ReplaceTablet: tablet 45c5e309389d402092f59c801923854a deleted and replaced by tablet 5eb896c8922e412c97e2cc4891298830
I20260810 07:53:01.165023 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:01.165092 368 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:01.166970 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:01.169719 1880 ts_tablet_manager.cc:1929] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:01.128482 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:53:01.174293 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:01.164004 1971 consensus_queue.cc:237] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.156606 1963 raft_consensus.cc:493] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.177680 851 tablet_service.cc:1511] Processing CreateTablet for tablet 999f3e79ee4b4c368b60b202735c2556 (DEFAULT_TABLE table=default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.178000 1963 raft_consensus.cc:515] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.179191 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 999f3e79ee4b4c368b60b202735c2556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.180248 646 tablet_service.cc:1511] Processing CreateTablet for tablet a86401e669fa4358ae33ce2249b3a98e (DEFAULT_TABLE table=default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.186612 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 4 got 3
I20260810 07:53:01.192210 1963 leader_election.cc:290] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:01.166569 1987 raft_consensus.cc:697] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:01.193866 1064 tablet_bootstrap.cc:654] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.194037 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_df5853b3486743289def86718544ef6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:01.194196 1987 consensus_queue.cc:237] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:01.195887 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df5853b3486743289def86718544ef6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.201476 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86401e669fa4358ae33ce2249b3a98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.194871 1874 ts_tablet_manager.cc:1929] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260810 07:53:01.198825 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:01.203579 890 raft_consensus.cc:2468] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:01.204752 1874 log.cc:1199] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/08520a2ac0f64c3abd534a32100839a4
I20260810 07:53:01.190826 368 tablet_service.cc:1511] Processing CreateTablet for tablet a86401e669fa4358ae33ce2249b3a98e (DEFAULT_TABLE table=default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.206069 1874 ts_tablet_manager.cc:1950] T 08520a2ac0f64c3abd534a32100839a4 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:01.206970 368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86401e669fa4358ae33ce2249b3a98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.206882 678 raft_consensus.cc:2468] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:01.213990 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:01.224450 792 leader_election.cc:304] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:01.225298 1753 raft_consensus.cc:2804] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:01.225775 1753 raft_consensus.cc:697] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:01.227013 1880 log.cc:1199] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/f9d3fcf79a104c47a6fd10fc34f7ce08
I20260810 07:53:01.226595 1753 consensus_queue.cc:237] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.227800 1880 ts_tablet_manager.cc:1950] T f9d3fcf79a104c47a6fd10fc34f7ce08 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:01.224931 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_4c5928f06794426aa64b4c1a6111d2bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.231889 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c5928f06794426aa64b4c1a6111d2bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.234411 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 170) took 1342 ms. Trace:
I20260810 07:53:01.235927 574 leader_election.cc:304] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:01.235522 1874 rpcz_store.cc:276] 0810 07:52:59.891945 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.062604 (+170659us) service_pool.cc:224] Handling call
0810 07:53:00.813984 (+751380us) ts_tablet_manager.cc:1151] Deleting tablet 08520a2ac0f64c3abd534a32100839a4
0810 07:53:00.828322 (+ 14338us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:01.034530 (+206208us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.194628 (+160098us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.234329 (+ 39701us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":712788}
W20260810 07:53:01.239583 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 45c5e309389d402092f59c801923854a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45c5e309389d402092f59c801923854a already in progress: creating tablet
I20260810 07:53:01.240818 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]) successfully deleted
I20260810 07:53:01.240883 1874 tablet_replica.cc:333] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:01.243161 639 tablet_service.cc:1511] Processing CreateTablet for tablet 92c95cd95c034d9a9205221d07ae8615 (DEFAULT_TABLE table=default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.243304 859 tablet_service.cc:1511] Processing CreateTablet for tablet a86401e669fa4358ae33ce2249b3a98e (DEFAULT_TABLE table=default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.244234 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c95cd95c034d9a9205221d07ae8615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.245009 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a86401e669fa4358ae33ce2249b3a98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.246062 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 149) took 1234 ms. Trace:
I20260810 07:53:01.246490 1880 rpcz_store.cc:276] 0810 07:53:00.011742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.012504 (+ 762us) service_pool.cc:224] Handling call
0810 07:53:00.709933 (+697429us) ts_tablet_manager.cc:1151] Deleting tablet f9d3fcf79a104c47a6fd10fc34f7ce08
0810 07:53:00.709997 (+ 64us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:01.068691 (+358694us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.169554 (+100863us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.245989 (+ 76435us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":696679}
I20260810 07:53:01.248193 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet f9d3fcf79a104c47a6fd10fc34f7ce08 (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
I20260810 07:53:01.257082 1874 raft_consensus.cc:2243] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:01.258292 371 tablet_service.cc:1511] Processing CreateTablet for tablet 92c95cd95c034d9a9205221d07ae8615 (DEFAULT_TABLE table=default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.259516 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c95cd95c034d9a9205221d07ae8615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.259975 1874 raft_consensus.cc:2272] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:01.260069 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3d1903c2423c4def8e00c8316402d0dc" } new_table_name: "default.Table_c5971a0e5bac4dbd907990ca708bd38c" modify_external_catalogs: true
I20260810 07:53:01.267242 1874 ts_tablet_manager.cc:1916] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:01.270206 1071 tablet_bootstrap.cc:492] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:01.270787 1071 ts_tablet_manager.cc:1403] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.392s user 0.021s sys 0.005s
I20260810 07:53:01.280375 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:01.281237 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:53:01.281925 32467 catalog_manager.cc:5719] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:01.289810 1071 raft_consensus.cc:359] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:01.283473 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260810 07:53:01.290647 1071 raft_consensus.cc:385] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:01.290918 1071 raft_consensus.cc:740] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:01.292052 1071 consensus_queue.cc:260] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.294833 1071 ts_tablet_manager.cc:1434] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260810 07:53:01.297014 1064 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:01.297510 1064 ts_tablet_manager.cc:1403] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.193s user 0.009s sys 0.008s
I20260810 07:53:01.303682 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:01.314723 369 tablet_service.cc:1511] Processing CreateTablet for tablet ebfc73a83a0f4048ad9dded62d074e19 (DEFAULT_TABLE table=default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.322037 377 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6e61aa223048f8bdc9eee07fa2066c (DEFAULT_TABLE table=default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.322261 1985 raft_consensus.cc:2804] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:01.322832 1985 raft_consensus.cc:493] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:01.323323 1985 raft_consensus.cc:3060] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:01.327116 376 tablet_service.cc:1511] Processing CreateTablet for tablet e6800738970d47dd8e7cba68945042aa (DEFAULT_TABLE table=default.Table_3d1903c2423c4def8e00c8316402d0dc [id=c8d56a9f72d541f780991aea52939a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.328173 367 tablet_service.cc:1511] Processing CreateTablet for tablet 1e629e89eb734ade82b3dcbf11c971f8 (DEFAULT_TABLE table=default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.330852 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:01.331454 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:01.332139 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 4 got 3
I20260810 07:53:01.334272 385 tablet_service.cc:1511] Processing CreateTablet for tablet 83b68614cd8444a08c7820a33c59442d (DEFAULT_TABLE table=default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.339226 380 tablet_service.cc:1511] Processing CreateTablet for tablet a61dbafb4f7444e690cea0d003d38f22 (DEFAULT_TABLE table=default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.344095 1064 raft_consensus.cc:359] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.344812 1956 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 178) took 1602 ms. Trace:
I20260810 07:53:01.345088 1064 raft_consensus.cc:385] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:01.345293 1071 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:01.345350 1064 raft_consensus.cc:740] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:01.346253 1064 consensus_queue.cc:260] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.348958 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6800738970d47dd8e7cba68945042aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.349020 1880 tablet_replica.cc:333] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:01.350046 1880 raft_consensus.cc:2243] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260810 07:53:01.314074 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e6800738970d47dd8e7cba68945042aa consensus state has no leader
I20260810 07:53:01.350517 382 tablet_service.cc:1511] Processing CreateTablet for tablet 23bd442cbd9542ab8a521ededc6b1e90 (DEFAULT_TABLE table=default.Table_f2a1788b7d16410e92f4494c0bfaa865 [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.351677 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bd442cbd9542ab8a521ededc6b1e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.355386 852 tablet_service.cc:1511] Processing CreateTablet for tablet e6800738970d47dd8e7cba68945042aa (DEFAULT_TABLE table=default.Table_3d1903c2423c4def8e00c8316402d0dc [id=c8d56a9f72d541f780991aea52939a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.356802 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfc73a83a0f4048ad9dded62d074e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.360333 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6e61aa223048f8bdc9eee07fa2066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.356884 866 tablet_service.cc:1511] Processing CreateTablet for tablet 1e629e89eb734ade82b3dcbf11c971f8 (DEFAULT_TABLE table=default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.358563 872 tablet_service.cc:1511] Processing CreateTablet for tablet a61dbafb4f7444e690cea0d003d38f22 (DEFAULT_TABLE table=default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.358796 855 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6e61aa223048f8bdc9eee07fa2066c (DEFAULT_TABLE table=default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.357937 856 tablet_service.cc:1511] Processing CreateTablet for tablet 83b68614cd8444a08c7820a33c59442d (DEFAULT_TABLE table=default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.357812 860 tablet_service.cc:1511] Processing CreateTablet for tablet ebfc73a83a0f4048ad9dded62d074e19 (DEFAULT_TABLE table=default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.365499 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e629e89eb734ade82b3dcbf11c971f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.369268 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83b68614cd8444a08c7820a33c59442d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.375110 1047 tablet_bootstrap.cc:654] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.378976 1064 ts_tablet_manager.cc:1434] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.081s user 0.004s sys 0.003s
I20260810 07:53:01.380208 1064 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:01.381328 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a61dbafb4f7444e690cea0d003d38f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.385130 1753 raft_consensus.cc:493] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.385715 1753 raft_consensus.cc:515] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.356709 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6800738970d47dd8e7cba68945042aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.394804 872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a61dbafb4f7444e690cea0d003d38f22. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.398959 1970 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.425s user 0.002s sys 0.010s
I20260810 07:53:01.401081 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfc73a83a0f4048ad9dded62d074e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.405155 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6e61aa223048f8bdc9eee07fa2066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.407641 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83b68614cd8444a08c7820a33c59442d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.409098 1064 tablet_bootstrap.cc:654] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.410543 1753 leader_election.cc:290] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:01.412148 1994 log.cc:927] Time spent T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.092s user 0.000s sys 0.001s
I20260810 07:53:01.412803 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e629e89eb734ade82b3dcbf11c971f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.412844 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:01.417122 792 leader_election.cc:343] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.429828 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_efee9f7cc3774e24baf8069ac53455ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.431620 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efee9f7cc3774e24baf8069ac53455ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.435729 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e1294c8e0ca349b9b06af3a9b62a88cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:01.345203 1956 rpcz_store.cc:276] 0810 07:52:59.742438 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:59.787021 (+ 44583us) service_pool.cc:224] Handling call
0810 07:53:01.344741 (+1557720us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:59.790228 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:52:59.790799 (+ 571us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:52:59.790832 (+ 33us) alter_schema_op.cc:66] Acquired schema lock
0810 07:52:59.790955 (+ 123us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:52:59.791109 (+ 154us) alter_schema_op.cc:128] START. Timestamp: 1786348379790997
0810 07:52:59.791143 (+ 34us) op_driver.cc:348] REPLICATION: starting
0810 07:52:59.791855 (+ 712us) log.cc:844] Serialized 173 byte log entry
0810 07:53:00.653554 (+861699us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.782178 (+128624us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:00.853522 (+ 71344us) log.cc:844] Serialized 5 byte log entry
0810 07:53:01.342493 (+488971us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.343323 (+ 830us) log.cc:844] Serialized 14 byte log entry
0810 07:53:01.344576 (+ 1253us) alter_schema_op.cc:73] Released schema lock
0810 07:53:01.344610 (+ 34us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":128357,"prepare.queue_time_us":2269,"prepare.run_cpu_time_us":3152,"prepare.run_wall_time_us":47053,"raft.queue_time_us":1376,"raft.run_cpu_time_us":1764,"raft.run_wall_time_us":1763,"replication_time_us":862289,"thread_start_us":173095,"threads_started":2,"wal-append.queue_time_us":35389}]]}
W20260810 07:53:01.437600 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1294c8e0ca349b9b06af3a9b62a88cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.452304 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:01.452867 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:01.439729 1985 raft_consensus.cc:515] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.454255 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:01.456211 1985 leader_election.cc:290] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:01.456629 1874 ts_tablet_manager.cc:1929] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:01.457093 1874 log.cc:1199] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:53:01.457926 1874 ts_tablet_manager.cc:1950] T cf45a33485e9463d89b5ce8affa0bd22 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:01.464852 1064 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:01.465263 1064 ts_tablet_manager.cc:1403] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.085s user 0.006s sys 0.008s
I20260810 07:53:01.467656 1064 raft_consensus.cc:359] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.468245 1064 raft_consensus.cc:385] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:01.468483 1064 raft_consensus.cc:740] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:01.470191 1064 consensus_queue.cc:260] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.471832 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 171) took 1326 ms. Trace:
I20260810 07:53:01.350514 1880 raft_consensus.cc:2272] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:01.479437 1880 ts_tablet_manager.cc:1916] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:01.485690 375 tablet_service.cc:1511] Processing CreateTablet for tablet ed74e441397344e09e924fb724ad55d6 (DEFAULT_TABLE table=default.Table_1ac5b694f50044beb1c28f2388c02f4a [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.489180 859 tablet_service.cc:1511] Processing CreateTablet for tablet 23bd442cbd9542ab8a521ededc6b1e90 (DEFAULT_TABLE table=default.Table_f2a1788b7d16410e92f4494c0bfaa865 [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.491281 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bd442cbd9542ab8a521ededc6b1e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.504446 1874 rpcz_store.cc:276] 0810 07:53:00.145361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.145531 (+ 170us) service_pool.cc:224] Handling call
0810 07:53:01.240620 (+1095089us) ts_tablet_manager.cc:1151] Deleting tablet cf45a33485e9463d89b5ce8affa0bd22
0810 07:53:01.240707 (+ 87us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:01.402508 (+161801us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.456424 (+ 53916us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.471718 (+ 15294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3876,"tablet-delete.queue_time_us":1094445}
I20260810 07:53:01.513760 376 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb66183211f45d4a756f70c5e331f87 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.514288 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_712d9cfefe684238a26898997e816b64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:01.515516 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb66183211f45d4a756f70c5e331f87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.516073 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_712d9cfefe684238a26898997e816b64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.521005 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed74e441397344e09e924fb724ad55d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.511482 851 tablet_service.cc:1511] Processing CreateTablet for tablet ed74e441397344e09e924fb724ad55d6 (DEFAULT_TABLE table=default.Table_1ac5b694f50044beb1c28f2388c02f4a [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.524494 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet cf45a33485e9463d89b5ce8affa0bd22 (table default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]) successfully deleted
I20260810 07:53:01.524711 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed74e441397344e09e924fb724ad55d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.525525 1874 tablet_replica.cc:333] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:01.526474 1874 raft_consensus.cc:2243] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:01.527077 1874 raft_consensus.cc:2272] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:01.515126 1064 ts_tablet_manager.cc:1434] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20260810 07:53:01.528998 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 1 got 0
I20260810 07:53:01.534417 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:01.535583 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]). Expected version 4 got 3
I20260810 07:53:01.541141 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:01.539685 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b1983fd910c94cb183eb1c99df54a1a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.545909 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1983fd910c94cb183eb1c99df54a1a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.546005 1064 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:01.545946 862 tablet_service.cc:1511] Processing CreateTablet for tablet 92c95cd95c034d9a9205221d07ae8615 (DEFAULT_TABLE table=default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.547282 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c95cd95c034d9a9205221d07ae8615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.551512 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:01.551729 1064 tablet_bootstrap.cc:654] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.553251 32469 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 219) took 1223 ms. Trace:
I20260810 07:53:01.554363 864 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:01.553584 32469 rpcz_store.cc:276] 0810 07:53:00.330163 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.338930 (+ 8767us) service_pool.cc:224] Handling call
0810 07:53:00.442408 (+103478us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:00.443043 (+ 635us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:00.443187 (+ 144us) catalog_manager.cc:2850] Locking tablets
0810 07:53:00.444382 (+ 1195us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:00.544472 (+100090us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.544498 (+ 26us) write_op.cc:454] Released schema lock
0810 07:53:00.544770 (+ 272us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:00.545002 (+ 232us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:01.551404 (+1006402us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:01.553191 (+ 1787us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:00.452963 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:00.453609 (+ 646us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:00.453641 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:53:00.453653 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:00.453973 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:00.454017 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:00.454049 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:00.454326 (+ 277us) tablet.cc:701] Row locks acquired
0810 07:53:00.454340 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:00.454451 (+ 111us) write_op.cc:270] Start()
0810 07:53:00.454546 (+ 95us) write_op.cc:276] Timestamp: P: 1786348380454364 usec, L: 0
0810 07:53:00.454563 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:00.455090 (+ 527us) log.cc:844] Serialized 762 byte log entry
0810 07:53:00.484307 (+ 29217us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.484923 (+ 616us) write_op.cc:301] APPLY: starting
0810 07:53:00.484981 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:00.485101 (+ 120us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:00.485211 (+ 110us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:00.485470 (+ 259us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:00.485624 (+ 154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:00.485673 (+ 49us) write_op.cc:312] APPLY: finished
0810 07:53:00.485857 (+ 184us) log.cc:844] Serialized 29 byte log entry
0810 07:53:00.486018 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.486091 (+ 73us) write_op.cc:454] Released schema lock
0810 07:53:00.486130 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":430,"apply.run_cpu_time_us":1401,"apply.run_wall_time_us":1402,"num_ops":2,"prepare.queue_time_us":6652,"prepare.run_cpu_time_us":2377,"prepare.run_wall_time_us":5329,"raft.queue_time_us":160,"raft.run_cpu_time_us":643,"raft.run_wall_time_us":641,"replication_time_us":29653}]]}
I20260810 07:53:01.556895 1874 ts_tablet_manager.cc:1916] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:01.566056 864 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb66183211f45d4a756f70c5e331f87 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.567674 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb66183211f45d4a756f70c5e331f87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.573904 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 540771us
I20260810 07:53:01.578287 849 tablet_service.cc:1511] Processing CreateTablet for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (DEFAULT_TABLE table=default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.579797 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2fe3153512b40a99caf3fb9b81fd7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.592902 367 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:01.594718 369 tablet_service.cc:1511] Processing CreateTablet for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (DEFAULT_TABLE table=default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.596025 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2fe3153512b40a99caf3fb9b81fd7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.596239 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f50ce5293cd74b41a97f9a6b66c60211" } new_table_name: "default.Table_ec9f1bb881a9463cbb641da255034aef" modify_external_catalogs: true
I20260810 07:53:01.598433 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:01.599040 894 raft_consensus.cc:3060] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:01.603174 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:01.606137 625 tablet_service.cc:1511] Processing CreateTablet for tablet e6800738970d47dd8e7cba68945042aa (DEFAULT_TABLE table=default.Table_3d1903c2423c4def8e00c8316402d0dc [id=c8d56a9f72d541f780991aea52939a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.607681 625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6800738970d47dd8e7cba68945042aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.609663 645 tablet_service.cc:1511] Processing CreateTablet for tablet ebfc73a83a0f4048ad9dded62d074e19 (DEFAULT_TABLE table=default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.611862 627 tablet_service.cc:1511] Processing CreateTablet for tablet 1e629e89eb734ade82b3dcbf11c971f8 (DEFAULT_TABLE table=default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:01.614089 792 leader_election.cc:343] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.614512 792 leader_election.cc:304] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:01.616046 1753 raft_consensus.cc:2749] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:01.617990 631 tablet_service.cc:1511] Processing CreateTablet for tablet 83b68614cd8444a08c7820a33c59442d (DEFAULT_TABLE table=default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.636847 1071 tablet_bootstrap.cc:654] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:01.644956 367 tablet_service.cc:1511] Processing CreateTablet for tablet 3c97418290e04f60884c721ded14495a (DEFAULT_TABLE table=default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.646155 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c97418290e04f60884c721ded14495a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.649448 894 raft_consensus.cc:2468] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:01.657522 860 tablet_service.cc:1511] Processing CreateTablet for tablet c43aeafc19a240e6991b69249b0516fa (DEFAULT_TABLE table=default.Table_c8825a5b51f241a1b6539a796438bbb4 [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:01.661103 32463 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6cd9484084294acfba9613fa49f369b2 has no consensus state
I20260810 07:53:01.665748 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43aeafc19a240e6991b69249b0516fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.664546 851 tablet_service.cc:1511] Processing CreateTablet for tablet 3c97418290e04f60884c721ded14495a (DEFAULT_TABLE table=default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.671274 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c97418290e04f60884c721ded14495a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.668277 32465 master_service.cc:960] ReplaceTablet: received request to replace tablet d7626d30f4574658bb226f2e915ee0de from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:01.604354 638 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6e61aa223048f8bdc9eee07fa2066c (DEFAULT_TABLE table=default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.673617 382 tablet_service.cc:1511] Processing CreateTablet for tablet c43aeafc19a240e6991b69249b0516fa (DEFAULT_TABLE table=default.Table_c8825a5b51f241a1b6539a796438bbb4 [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.674646 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43aeafc19a240e6991b69249b0516fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.677976 624 tablet_service.cc:1511] Processing CreateTablet for tablet a61dbafb4f7444e690cea0d003d38f22 (DEFAULT_TABLE table=default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.461066 1970 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 174) took 1802 ms. Trace:
I20260810 07:53:01.682065 376 tablet_service.cc:1511] Processing CreateTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e (DEFAULT_TABLE table=default.Table_868b99bda3b04eafab388ae9cbf490fd [id=9e2911100a1c42d2a467f6156a8f6bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.683104 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.676556 1926 raft_consensus.cc:493] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.685040 1926 raft_consensus.cc:515] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.687153 1926 leader_election.cc:290] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:01.689340 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:01.690459 1071 tablet_bootstrap.cc:492] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
W20260810 07:53:01.691359 792 leader_election.cc:343] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.576222 573 leader_election.cc:304] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:01.693325 1071 ts_tablet_manager.cc:1403] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.348s user 0.012s sys 0.000s
I20260810 07:53:01.696467 1071 raft_consensus.cc:359] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.697147 1071 raft_consensus.cc:385] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:01.697417 1071 raft_consensus.cc:740] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:01.698230 1071 consensus_queue.cc:260] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.701269 852 tablet_service.cc:1511] Processing CreateTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e (DEFAULT_TABLE table=default.Table_868b99bda3b04eafab388ae9cbf490fd [id=9e2911100a1c42d2a467f6156a8f6bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.702481 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.703763 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfc73a83a0f4048ad9dded62d074e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.705200 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_bb6691e602074986a5e5ecee6fac625e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.707340 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb6691e602074986a5e5ecee6fac625e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.713832 1926 raft_consensus.cc:493] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.714301 1926 raft_consensus.cc:515] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.714799 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_2a1f8373fc464a359e73cd83b7381720"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.716461 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a1f8373fc464a359e73cd83b7381720 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.718142 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:01.723678 792 leader_election.cc:343] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.716598 1926 leader_election.cc:290] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:01.729353 635 tablet_service.cc:1511] Processing CreateTablet for tablet 23bd442cbd9542ab8a521ededc6b1e90 (DEFAULT_TABLE table=default.Table_f2a1788b7d16410e92f4494c0bfaa865 [id=7e8c44d3660746d69e0982f1cf2dc80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.730242 1987 raft_consensus.cc:493] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.730993 1987 raft_consensus.cc:515] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.737061 1880 ts_tablet_manager.cc:1929] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:01.742074 630 tablet_service.cc:1511] Processing CreateTablet for tablet ed74e441397344e09e924fb724ad55d6 (DEFAULT_TABLE table=default.Table_1ac5b694f50044beb1c28f2388c02f4a [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.745428 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e629e89eb734ade82b3dcbf11c971f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.754736 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:01.755334 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:01.679025 1970 rpcz_store.cc:276] 0810 07:52:59.658497 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:59.658613 (+ 116us) service_pool.cc:224] Handling call
0810 07:53:01.460978 (+1802365us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:52:59.718590 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:52:59.719175 (+ 585us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:52:59.719207 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0810 07:52:59.719354 (+ 147us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:52:59.719526 (+ 172us) alter_schema_op.cc:128] START. Timestamp: 1786348379719422
0810 07:52:59.719557 (+ 31us) op_driver.cc:348] REPLICATION: starting
0810 07:52:59.720304 (+ 747us) log.cc:844] Serialized 173 byte log entry
0810 07:53:00.761650 (+1041346us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.841928 (+ 80278us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:01.075183 (+233255us) log.cc:844] Serialized 5 byte log entry
0810 07:53:01.459542 (+384359us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.460605 (+ 1063us) log.cc:844] Serialized 14 byte log entry
0810 07:53:01.460755 (+ 150us) alter_schema_op.cc:73] Released schema lock
0810 07:53:01.460785 (+ 30us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":80069,"prepare.queue_time_us":58897,"prepare.run_cpu_time_us":2004,"prepare.run_wall_time_us":110324,"raft.queue_time_us":1180,"raft.run_cpu_time_us":1153,"raft.run_wall_time_us":1854,"replication_time_us":1041959,"thread_start_us":476160,"threads_started":3,"wal-append.queue_time_us":314206}]]}
W20260810 07:53:01.756387 573 raft_consensus.cc:2686] T 449359c8e9fa4756ba149f3dce6e0ce7 P b5bfa873accc428ab48ac19e1207f88d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260810 07:53:01.760885 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 185103us
W20260810 07:53:01.761905 573 consensus_peers.cc:597] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:01.762593 573 leader_election.cc:343] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.767340 1671 consensus_queue.cc:1048] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:01.782562 574 leader_election.cc:304] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:01.790071 32469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_903e7df2a6cc4ebdb26c728e4c047cfc" } modify_external_catalogs: true
I20260810 07:53:01.790719 32469 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_903e7df2a6cc4ebdb26c728e4c047cfc" } modify_external_catalogs: true
I20260810 07:53:01.794217 1071 ts_tablet_manager.cc:1434] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.101s user 0.005s sys 0.000s
I20260810 07:53:01.784375 1987 leader_election.cc:290] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:01.789031 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6e61aa223048f8bdc9eee07fa2066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.797024 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bd442cbd9542ab8a521ededc6b1e90. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:01.802112 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 116450us
I20260810 07:53:01.802376 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed74e441397344e09e924fb724ad55d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.737927 1880 log.cc:1199] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/4ec00a494816409684205c7f4435a079
I20260810 07:53:01.804906 1971 raft_consensus.cc:2804] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:01.805110 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a61dbafb4f7444e690cea0d003d38f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.805601 1880 ts_tablet_manager.cc:1950] T 4ec00a494816409684205c7f4435a079 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:01.807752 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 153) took 1783 ms. Trace:
I20260810 07:53:01.808084 1880 rpcz_store.cc:276] 0810 07:53:00.023895 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.024042 (+ 147us) service_pool.cc:224] Handling call
0810 07:53:01.295242 (+1271200us) ts_tablet_manager.cc:1151] Deleting tablet 4ec00a494816409684205c7f4435a079
0810 07:53:01.319710 (+ 24468us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b5800000f98. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c64327981 00007f9c64467521 00007f9c64467b52 00007f9c6d3d51d9 00007f9c6c5ed4e8 00007f9c6c6b3aa9 00007f9c6c6b40a3 00007f9c6c6e50e0 00007f9c6c6cb64d 00007f9c6c6cb5f9 00007f9c6c6cb589
0810 07:53:01.348614 (+ 28904us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:01.610659 (+262045us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.736790 (+126131us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.807686 (+ 70896us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8079744,"tablet-delete.queue_time_us":1269903}
I20260810 07:53:01.809460 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 4ec00a494816409684205c7f4435a079 (table default.Table_53683cfe39cb4ac08b9cd39e9e0b6da7 [id=985c6c2d9638454f90dc084e43ebf4d8]) successfully deleted
I20260810 07:53:01.813176 1880 tablet_replica.cc:333] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:01.814069 1880 raft_consensus.cc:2243] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:01.814566 1880 raft_consensus.cc:2272] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:01.824029 1071 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:01.832058 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_6b73ad51b4db4578be0a78f07ed60644"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:01.833918 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b73ad51b4db4578be0a78f07ed60644 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:01.854208 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:01.858836 1880 ts_tablet_manager.cc:1916] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:01.862519 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:01.863343 895 raft_consensus.cc:2468] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:01.865864 1071 tablet_bootstrap.cc:654] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:01.867245 792 leader_election.cc:343] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.867770 792 leader_election.cc:304] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:01.869056 1926 raft_consensus.cc:2749] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:01.883635 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_f2a1788b7d16410e92f4494c0bfaa865" } new_table_name: "default.Table_b804194501bd4439adcce524793449cd" modify_external_catalogs: true
I20260810 07:53:01.886991 1753 raft_consensus.cc:493] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.887558 1753 raft_consensus.cc:515] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.889734 1753 leader_election.cc:290] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:01.894093 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:01.894570 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:01.906623 1946 raft_consensus.cc:493] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:01.907269 1946 raft_consensus.cc:515] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:01.908547 792 leader_election.cc:343] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:01.908944 792 leader_election.cc:304] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:01.909617 1926 raft_consensus.cc:2749] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:01.909840 1946 leader_election.cc:290] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:01.805784 1971 raft_consensus.cc:697] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:01.915380 1971 consensus_queue.cc:237] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:01.917760 623 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb66183211f45d4a756f70c5e331f87 (DEFAULT_TABLE table=default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:01.910249 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260810 07:53:01.925098 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:01.927012 633 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:01.928447 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 81693.1us
W20260810 07:53:01.929443 573 leader_election.cc:343] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:01.930851 573 consensus_peers.cc:597] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:01.897804 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83b68614cd8444a08c7820a33c59442d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:01.962064 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:01.963161 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 52925.2us
I20260810 07:53:02.011276 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a04e29d3a49b4bd891be79e0fad6ec51" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:02.012609 1880 ts_tablet_manager.cc:1929] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:02.029389 1880 log.cc:1199] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:53:02.030113 1880 ts_tablet_manager.cc:1950] T 92e2678ab165486dbbb69c7c183a7fa4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.034201 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:02.048534 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 154) took 1987 ms. Trace:
I20260810 07:53:02.048969 1880 rpcz_store.cc:276] 0810 07:53:00.061612 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.061741 (+ 129us) service_pool.cc:224] Handling call
0810 07:53:01.813008 (+1751267us) ts_tablet_manager.cc:1151] Deleting tablet 92e2678ab165486dbbb69c7c183a7fa4
0810 07:53:01.813059 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:01.935918 (+122859us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.012477 (+ 76559us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.048462 (+ 35985us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1750594}
I20260810 07:53:02.057873 2039 tablet.cc:1724] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:01.856225 32468 catalog_manager.cc:5719] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:02.075863 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:02.076975 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:02.080282 622 tablet_service.cc:1511] Processing CreateTablet for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (DEFAULT_TABLE table=default.Table_c56854332dfd4628a280b1d87bf7a2cb [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:01.913766 1064 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:02.099507 1064 ts_tablet_manager.cc:1403] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.554s user 0.007s sys 0.004s
I20260810 07:53:02.102545 1064 raft_consensus.cc:359] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.112401 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb66183211f45d4a756f70c5e331f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.122269 1064 raft_consensus.cc:385] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.122605 1064 raft_consensus.cc:740] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.123310 1064 consensus_queue.cc:260] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.123677 620 tablet_service.cc:1511] Processing CreateTablet for tablet 3c97418290e04f60884c721ded14495a (DEFAULT_TABLE table=default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:02.114203 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 205671us
I20260810 07:53:02.134730 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd22d1029ff249a0a9e05862293261ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:02.143224 574 leader_election.cc:304] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:02.060214 2042 tablet.cc:1724] T 9369baf30ab64646b7bc042c6da1987b P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260810 07:53:02.062091 1880 tablet_replica.cc:333] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.150345 1880 raft_consensus.cc:2243] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.150951 1880 raft_consensus.cc:2272] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.072979 1946 raft_consensus.cc:493] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:02.152964 1946 raft_consensus.cc:515] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.154989 1946 leader_election.cc:290] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:02.091037 1071 tablet_bootstrap.cc:492] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.155611 1071 ts_tablet_manager.cc:1403] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.332s user 0.020s sys 0.008s
I20260810 07:53:02.153689 619 tablet_service.cc:1511] Processing CreateTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e (DEFAULT_TABLE table=default.Table_868b99bda3b04eafab388ae9cbf490fd [id=9e2911100a1c42d2a467f6156a8f6bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:02.156956 642 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 198) took 1047 ms. Trace:
I20260810 07:53:02.157294 642 rpcz_store.cc:276] 0810 07:53:01.109372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.109523 (+ 151us) service_pool.cc:224] Handling call
0810 07:53:01.110828 (+ 1305us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.110951 (+ 123us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:01.266931 (+155980us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.978234 (+711303us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.156765 (+178531us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:02.157728 309 leader_election.cc:343] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:53:02.157931 1071 raft_consensus.cc:359] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.158283 1946 raft_consensus.cc:493] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:02.158573 1071 raft_consensus.cc:385] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.158852 644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 199) took 1022 ms. Trace:
I20260810 07:53:02.158828 1071 raft_consensus.cc:740] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.158772 1946 raft_consensus.cc:515] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.159143 644 rpcz_store.cc:276] 0810 07:53:01.136155 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.136295 (+ 140us) service_pool.cc:224] Handling call
0810 07:53:01.140918 (+ 4623us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.141013 (+ 95us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:01.188547 (+ 47534us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:01.895353 (+706806us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.158689 (+263336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.161367 1671 raft_consensus.cc:2804] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:02.161305 1946 leader_election.cc:290] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:02.161744 1671 raft_consensus.cc:493] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:02.162071 1671 raft_consensus.cc:3060] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:02.164269 309 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
I20260810 07:53:02.168108 1064 ts_tablet_manager.cc:1434] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.068s user 0.007s sys 0.000s
I20260810 07:53:02.169195 1064 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:02.173245 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2fe3153512b40a99caf3fb9b81fd7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.177399 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd22d1029ff249a0a9e05862293261ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:02.178242 1880 ts_tablet_manager.cc:1916] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.190371 620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c97418290e04f60884c721ded14495a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.193224 1874 ts_tablet_manager.cc:1929] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:02.159497 1071 consensus_queue.cc:260] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:02.199942 574 consensus_peers.cc:597] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:02.200965 1071 ts_tablet_manager.cc:1434] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
I20260810 07:53:02.207449 412 raft_consensus.cc:3060] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:02.207742 573 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:02.209545 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet fc12205a097c496cba5b777b98f27914 consensus state has no leader
W20260810 07:53:02.208624 573 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:02.218652 573 leader_election.cc:343] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:02.223153 1071 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:02.227667 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.232703 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:02.233835 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:02.234735 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 5e4b3ca46c9c44559761e25bcde3e927
W20260810 07:53:02.193421 646 consensus_meta.cc:287] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: Time spent flushing consensus metadata: real 0.631s user 0.013s sys 0.003s
I20260810 07:53:02.236338 32464 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 217) took 2261 ms. Trace:
I20260810 07:53:02.236625 32464 rpcz_store.cc:276] 0810 07:52:59.974782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.143767 (+168985us) service_pool.cc:224] Handling call
0810 07:53:00.145585 (+ 1818us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:00.146289 (+ 704us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:00.146426 (+ 137us) catalog_manager.cc:2850] Locking tablets
0810 07:53:00.505787 (+359361us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:00.802885 (+297098us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:00.803154 (+ 269us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:02.234641 (+1431487us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:02.236266 (+ 1625us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:00.513018 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:00.513439 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:00.513465 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:00.513479 (+ 14us) tablet.cc:662] Decoding operations
0810 07:53:00.513813 (+ 334us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:00.513854 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:00.513877 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:00.514138 (+ 261us) tablet.cc:701] Row locks acquired
0810 07:53:00.514153 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:00.514266 (+ 113us) write_op.cc:270] Start()
0810 07:53:00.514345 (+ 79us) write_op.cc:276] Timestamp: P: 1786348380514175 usec, L: 0
0810 07:53:00.514361 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:00.721644 (+207283us) log.cc:844] Serialized 765 byte log entry
0810 07:53:00.723125 (+ 1481us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf964951 00007f30cf9f7208 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1 00007f30cfe8a50d 00000000005bd8c4 00000000005bd6f9
0810 07:53:00.799838 (+ 76713us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.800889 (+ 1051us) write_op.cc:301] APPLY: starting
0810 07:53:00.800947 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:00.801053 (+ 106us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:00.801069 (+ 16us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:00.801345 (+ 276us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:00.801527 (+ 182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:00.801576 (+ 49us) write_op.cc:312] APPLY: finished
0810 07:53:00.801781 (+ 205us) log.cc:844] Serialized 29 byte log entry
0810 07:53:00.801983 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.802065 (+ 82us) write_op.cc:454] Released schema lock
0810 07:53:00.802141 (+ 76us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:00.802422 (+ 281us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.802477 (+ 55us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":870,"apply.run_cpu_time_us":1493,"apply.run_wall_time_us":1492,"num_ops":2,"prepare.queue_time_us":5261,"prepare.run_cpu_time_us":4442,"prepare.run_wall_time_us":210327,"raft.queue_time_us":171,"raft.run_cpu_time_us":1068,"raft.run_wall_time_us":1069,"replication_time_us":285375,"spinlock_wait_cycles":475386112}]]}
I20260810 07:53:02.256778 1880 ts_tablet_manager.cc:1929] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20260810 07:53:02.257215 1880 log.cc:1199] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/08520a2ac0f64c3abd534a32100839a4
I20260810 07:53:02.257982 1880 ts_tablet_manager.cc:1950] T 08520a2ac0f64c3abd534a32100839a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.260145 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 161) took 2152 ms. Trace:
I20260810 07:53:02.264539 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:02.279246 639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 203) took 1036 ms. Trace:
I20260810 07:53:02.279752 639 rpcz_store.cc:276] 0810 07:53:01.242380 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.242497 (+ 117us) service_pool.cc:224] Handling call
0810 07:53:01.243786 (+ 1289us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.243872 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:01.846887 (+603015us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.278551 (+431664us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.279061 (+ 510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.281270 1874 log.cc:1199] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:02.281517 646 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 202) took 1102 ms. Trace:
I20260810 07:53:02.282302 1874 ts_tablet_manager.cc:1950] T c1faed468bfa4d1ba5d92c0b9b194781 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:02.281875 646 rpcz_store.cc:276] 0810 07:53:01.179480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.179609 (+ 129us) service_pool.cc:224] Handling call
0810 07:53:01.200911 (+ 21302us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5800000f98. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22b54e6981 00007f22b5626521 00007f22b56263d2 00007f22be56afb9 00007f22bd7ac640 00007f22bd891641 00007f22bd86e8d3 00007f22bd81d210 00007f22bbda9bf4 00007f22bbda9b82 00007f22bbda9ab1
0810 07:53:01.201031 (+ 120us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.201112 (+ 81us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:01.218888 (+ 17776us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.280955 (+1062067us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.281448 (+ 493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38570880}
I20260810 07:53:02.283396 633 tablet_service.cc:1511] Processing CreateTablet for tablet c43aeafc19a240e6991b69249b0516fa (DEFAULT_TABLE table=default.Table_c8825a5b51f241a1b6539a796438bbb4 [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:02.205385 792 leader_election.cc:343] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:02.265403 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:02.284380 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 79041us
I20260810 07:53:02.284613 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43aeafc19a240e6991b69249b0516fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.285506 1753 consensus_queue.cc:1048] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:02.289131 792 leader_election.cc:343] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:02.289592 792 leader_election.cc:304] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:02.290323 1753 raft_consensus.cc:2749] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:02.260483 1880 rpcz_store.cc:276] 0810 07:53:00.107597 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.115196 (+ 7599us) service_pool.cc:224] Handling call
0810 07:53:02.061874 (+1946678us) ts_tablet_manager.cc:1151] Deleting tablet 08520a2ac0f64c3abd534a32100839a4
0810 07:53:02.061937 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.218522 (+156585us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.256607 (+ 38085us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.260076 (+ 3469us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1945896}
I20260810 07:53:02.294184 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 08520a2ac0f64c3abd534a32100839a4 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]) successfully deleted
I20260810 07:53:02.295107 1880 tablet_replica.cc:333] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.295881 1880 raft_consensus.cc:2243] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.296080 1064 tablet_bootstrap.cc:654] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.296324 1880 raft_consensus.cc:2272] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260810 07:53:02.300681 32463 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 consensus state has no leader
I20260810 07:53:02.302225 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_ec2bd2c8af504b6c94376bc07ac5946b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:02.304184 1880 ts_tablet_manager.cc:1916] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:02.304140 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec2bd2c8af504b6c94376bc07ac5946b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.338196 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d2feab5fe7f043538f85acd989659d91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:02.339797 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2feab5fe7f043538f85acd989659d91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.346443 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 172) took 2200 ms. Trace:
I20260810 07:53:02.346892 1874 rpcz_store.cc:276] 0810 07:53:00.146537 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.146642 (+ 105us) service_pool.cc:224] Handling call
0810 07:53:01.525360 (+1378718us) ts_tablet_manager.cc:1151] Deleting tablet c1faed468bfa4d1ba5d92c0b9b194781
0810 07:53:01.525406 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.096854 (+571448us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.193076 (+ 96222us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.346371 (+153295us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1374394}
I20260810 07:53:02.348523 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet c1faed468bfa4d1ba5d92c0b9b194781 (table default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]) successfully deleted
I20260810 07:53:02.349733 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 173) took 2202 ms. Trace:
I20260810 07:53:02.351311 1874 rpcz_store.cc:276] 0810 07:53:00.147162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.151074 (+ 3912us) service_pool.cc:224] Handling call
0810 07:53:02.349336 (+2198262us) ts_tablet_manager.cc:1151] Deleting tablet 371fa2534984499b95c0945af358e8fc
0810 07:53:02.349389 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.349670 (+ 281us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2197664}
I20260810 07:53:02.361931 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:02.363729 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:02.396908 1600 consensus_queue.cc:1048] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:02.403136 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet d7626d30f4574658bb226f2e915ee0de
I20260810 07:53:02.404597 32465 catalog_manager.cc:6807] ReplaceTablet: tablet d7626d30f4574658bb226f2e915ee0de deleted and replaced by tablet c47792551bb243b7b3f6de904c17c47b
I20260810 07:53:02.409559 382 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:02.410043 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_6fb07c2480e74dfc9cd95d724ec163cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:02.411662 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fb07c2480e74dfc9cd95d724ec163cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.365952 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:02.407536 862 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:02.430128 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 371fa2534984499b95c0945af358e8fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 371fa2534984499b95c0945af358e8fc already in progress: creating tablet
I20260810 07:53:02.431565 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 175) took 1276 ms. Trace:
I20260810 07:53:02.435581 1600 consensus_queue.cc:1048] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:02.442495 32465 master_service.cc:960] ReplaceTablet: received request to replace tablet 213fa4838fe847de87091cfba97eaedc from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:02.452490 1071 tablet_bootstrap.cc:700] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent opening tablet: real 0.117s user 0.003s sys 0.001s
I20260810 07:53:02.459764 1071 tablet_bootstrap.cc:654] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.476707 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_98bd794f5968403fb88241d8e84ae242"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:02.478973 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98bd794f5968403fb88241d8e84ae242 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.483084 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:02.485532 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
W20260810 07:53:02.496182 2042 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.346s user 0.003s sys 0.007s
W20260810 07:53:02.509289 792 consensus_peers.cc:597] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:02.516168 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:02.525892 1880 ts_tablet_manager.cc:1929] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:02.526285 1880 log.cc:1199] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/338e9b0e89d44295973d4003abd36649
I20260810 07:53:02.526963 1880 ts_tablet_manager.cc:1950] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.528177 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 152) took 2504 ms. Trace:
I20260810 07:53:02.528486 1880 rpcz_store.cc:276] 0810 07:53:00.023557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.169117 (+145560us) service_pool.cc:224] Handling call
0810 07:53:02.294928 (+2125811us) ts_tablet_manager.cc:1151] Deleting tablet 338e9b0e89d44295973d4003abd36649
0810 07:53:02.294976 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.457448 (+162472us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.525762 (+ 68314us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.528114 (+ 2352us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2082813}
I20260810 07:53:02.529793 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 165) took 1903 ms. Trace:
I20260810 07:53:02.530095 1880 rpcz_store.cc:276] 0810 07:53:00.626190 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.626290 (+ 100us) service_pool.cc:224] Handling call
0810 07:53:02.529432 (+1903142us) ts_tablet_manager.cc:1151] Deleting tablet 371fa2534984499b95c0945af358e8fc
0810 07:53:02.529470 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.529743 (+ 273us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1902645}
I20260810 07:53:02.530484 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 338e9b0e89d44295973d4003abd36649 (table default.Table_fce89a2685ea4f499401deb1152927b9 [id=5b11481adea645a0a736ce82b79ef800]) successfully deleted
I20260810 07:53:02.531023 1880 tablet_replica.cc:333] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
W20260810 07:53:02.530869 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 65490.4us
I20260810 07:53:02.531742 1880 raft_consensus.cc:2243] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:53:02.531747 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 371fa2534984499b95c0945af358e8fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 371fa2534984499b95c0945af358e8fc already in progress: creating tablet
I20260810 07:53:02.532227 1880 raft_consensus.cc:2272] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.534044 1880 ts_tablet_manager.cc:1916] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.545465 32463 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d0c496eafbd142de89df899f2a939e6f" } modify_external_catalogs: true
I20260810 07:53:02.546038 32463 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d0c496eafbd142de89df899f2a939e6f" } modify_external_catalogs: true
I20260810 07:53:02.563185 1880 ts_tablet_manager.cc:1929] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:02.563556 1880 log.cc:1199] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:53:02.564254 1880 ts_tablet_manager.cc:1950] T cf45a33485e9463d89b5ce8affa0bd22 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.566187 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 163) took 1944 ms. Trace:
I20260810 07:53:02.566473 1880 rpcz_store.cc:276] 0810 07:53:00.621863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.625099 (+ 3236us) service_pool.cc:224] Handling call
0810 07:53:02.530882 (+1905783us) ts_tablet_manager.cc:1151] Deleting tablet cf45a33485e9463d89b5ce8affa0bd22
0810 07:53:02.530918 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.556317 (+ 25399us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.563060 (+ 6743us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.566128 (+ 3068us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1873004}
I20260810 07:53:02.567695 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet cf45a33485e9463d89b5ce8affa0bd22 (table default.Table_746adafe126742f79e73baaf7b1fcfa7 [id=732d85303bc4488e859727fab0fdeb2e]) successfully deleted
I20260810 07:53:02.568555 1880 tablet_replica.cc:333] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.569260 1047 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:02.569319 1880 raft_consensus.cc:2243] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.569751 1880 raft_consensus.cc:2272] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.569811 1047 ts_tablet_manager.cc:1403] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 1.719s user 0.025s sys 0.062s
I20260810 07:53:02.573917 1047 raft_consensus.cc:359] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.574573 1047 raft_consensus.cc:385] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.574839 1047 raft_consensus.cc:740] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.575309 1064 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:02.575883 1880 ts_tablet_manager.cc:1916] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.575500 1047 consensus_queue.cc:260] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:02.579097 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 68704.6us
I20260810 07:53:02.582180 1071 tablet_bootstrap.cc:492] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.582618 1071 ts_tablet_manager.cc:1403] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.360s user 0.008s sys 0.005s
I20260810 07:53:02.590081 1880 ts_tablet_manager.cc:1929] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:02.590466 1880 log.cc:1199] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/09712cfebab34e56ba844e7aa0e8d152
I20260810 07:53:02.591109 1880 ts_tablet_manager.cc:1950] T 09712cfebab34e56ba844e7aa0e8d152 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.593077 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 164) took 1967 ms. Trace:
I20260810 07:53:02.593402 1880 rpcz_store.cc:276] 0810 07:53:00.625930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.638352 (+ 12422us) service_pool.cc:224] Handling call
0810 07:53:02.568412 (+1930060us) ts_tablet_manager.cc:1151] Deleting tablet 09712cfebab34e56ba844e7aa0e8d152
0810 07:53:02.568452 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.583564 (+ 15112us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.589968 (+ 6404us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.592996 (+ 3028us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1903257}
I20260810 07:53:02.594681 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 09712cfebab34e56ba844e7aa0e8d152 (table default.Table_db3c7e150df447dab749a578605e8d4c [id=5fe5c44770974b8eb4f3b55096d25eeb]) successfully deleted
I20260810 07:53:02.595638 1880 tablet_replica.cc:333] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.597352 1064 ts_tablet_manager.cc:1403] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.428s user 0.010s sys 0.005s
W20260810 07:53:02.605823 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 166147us
I20260810 07:53:02.606931 1880 raft_consensus.cc:2243] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.607492 1880 raft_consensus.cc:2272] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.609308 1880 ts_tablet_manager.cc:1916] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.623592 1880 ts_tablet_manager.cc:1929] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:02.624045 1880 log.cc:1199] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/b2bfffdd01524eada0a85c1a525325a0
I20260810 07:53:02.624809 1880 ts_tablet_manager.cc:1950] T b2bfffdd01524eada0a85c1a525325a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.625589 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 162) took 2004 ms. Trace:
I20260810 07:53:02.432183 1874 rpcz_store.cc:276] 0810 07:53:01.155343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.155514 (+ 171us) service_pool.cc:224] Handling call
0810 07:53:02.431120 (+1275606us) ts_tablet_manager.cc:1151] Deleting tablet 0098aaa7be644c9885167dc1ee4929b5
0810 07:53:02.431192 (+ 72us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.431504 (+ 312us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1274703}
I20260810 07:53:02.626422 1671 raft_consensus.cc:515] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:02.627676 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 0098aaa7be644c9885167dc1ee4929b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0098aaa7be644c9885167dc1ee4929b5 already in progress: creating tablet
I20260810 07:53:02.629431 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 176) took 1466 ms. Trace:
I20260810 07:53:02.629990 1874 rpcz_store.cc:276] 0810 07:53:01.162561 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.164856 (+ 2295us) service_pool.cc:224] Handling call
0810 07:53:02.628933 (+1464077us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:02.628996 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.629343 (+ 347us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1463392}
W20260810 07:53:02.631946 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 45c5e309389d402092f59c801923854a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45c5e309389d402092f59c801923854a already in progress: creating tablet
I20260810 07:53:02.633266 1874 tablet_replica.cc:333] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:02.634801 1874 raft_consensus.cc:2243] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.635738 1874 raft_consensus.cc:2272] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.625944 1880 rpcz_store.cc:276] 0810 07:53:00.621601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.703834 (+ 82233us) service_pool.cc:224] Handling call
0810 07:53:02.595486 (+1891652us) ts_tablet_manager.cc:1151] Deleting tablet b2bfffdd01524eada0a85c1a525325a0
0810 07:53:02.595531 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.617310 (+ 21779us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.623486 (+ 6176us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.625537 (+ 2051us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1890781}
I20260810 07:53:02.638959 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet b2bfffdd01524eada0a85c1a525325a0 (table default.Table_97132925a8444a3d9c702a3ae7cbb209 [id=833396f0e1d74f589d03c1c918dd843a]) successfully deleted
I20260810 07:53:02.640130 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 168) took 1482 ms. Trace:
I20260810 07:53:02.640509 1880 rpcz_store.cc:276] 0810 07:53:01.157606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.157767 (+ 161us) service_pool.cc:224] Handling call
0810 07:53:02.639797 (+1482030us) ts_tablet_manager.cc:1151] Deleting tablet 0098aaa7be644c9885167dc1ee4929b5
0810 07:53:02.639837 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.640073 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1481358}
W20260810 07:53:02.641841 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 0098aaa7be644c9885167dc1ee4929b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0098aaa7be644c9885167dc1ee4929b5 already in progress: creating tablet
I20260810 07:53:02.642735 1071 raft_consensus.cc:359] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.643085 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 169) took 1478 ms. Trace:
I20260810 07:53:02.644691 1880 rpcz_store.cc:276] 0810 07:53:01.164641 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.164885 (+ 244us) service_pool.cc:224] Handling call
0810 07:53:02.642632 (+1477747us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:02.642672 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.643032 (+ 360us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1477030}
W20260810 07:53:02.649406 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 45c5e309389d402092f59c801923854a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45c5e309389d402092f59c801923854a already in progress: creating tablet
I20260810 07:53:02.650447 1880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 181) took 1092 ms. Trace:
I20260810 07:53:02.650789 1880 rpcz_store.cc:276] 0810 07:53:01.557897 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.592696 (+ 34799us) service_pool.cc:224] Handling call
0810 07:53:02.650127 (+1057431us) ts_tablet_manager.cc:1151] Deleting tablet dd22d1029ff249a0a9e05862293261ff
0810 07:53:02.650166 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:02.650395 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1056436}
W20260810 07:53:02.652124 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet dd22d1029ff249a0a9e05862293261ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd22d1029ff249a0a9e05862293261ff already in progress: creating tablet
I20260810 07:53:02.653062 1880 tablet_replica.cc:333] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.653476 1671 leader_election.cc:290] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:02.643426 1071 raft_consensus.cc:385] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.654134 1071 raft_consensus.cc:740] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.654407 1985 consensus_queue.cc:1048] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.054s
I20260810 07:53:02.653775 1880 raft_consensus.cc:2243] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.657123 1880 raft_consensus.cc:2272] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.659601 631 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 208) took 1046 ms. Trace:
I20260810 07:53:02.659330 1071 consensus_queue.cc:260] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.662914 1071 ts_tablet_manager.cc:1434] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.080s user 0.003s sys 0.005s
I20260810 07:53:02.672828 1880 ts_tablet_manager.cc:1916] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.679571 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:02.681177 1071 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:02.689862 1071 tablet_bootstrap.cc:654] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:02.691958 573 consensus_peers.cc:597] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:02.692581 631 rpcz_store.cc:276] 0810 07:53:01.613247 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.617080 (+ 3833us) service_pool.cc:224] Handling call
0810 07:53:01.677566 (+ 60486us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.709351 (+ 31785us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.521142 (+811791us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.658880 (+137738us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.659527 (+ 647us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:02.704926 630 consensus_meta.cc:287] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: Time spent flushing consensus metadata: real 0.585s user 0.003s sys 0.017s
W20260810 07:53:02.711396 573 leader_election.cc:343] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:02.711705 1880 ts_tablet_manager.cc:1929] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:02.712112 1880 log.cc:1199] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/5e4b3ca46c9c44559761e25bcde3e927
I20260810 07:53:02.712914 1880 ts_tablet_manager.cc:1950] T 5e4b3ca46c9c44559761e25bcde3e927 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.715427 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 5e4b3ca46c9c44559761e25bcde3e927 (table default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]) successfully deleted
I20260810 07:53:02.717458 624 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 209) took 1096 ms. Trace:
I20260810 07:53:02.718245 624 rpcz_store.cc:276] 0810 07:53:01.621326 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.621635 (+ 309us) service_pool.cc:224] Handling call
0810 07:53:01.768799 (+147164us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.768960 (+ 161us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.517820 (+748860us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.712959 (+195139us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.717338 (+ 4379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616640}
I20260810 07:53:02.723831 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 213fa4838fe847de87091cfba97eaedc
I20260810 07:53:02.725665 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:02.725796 627 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 207) took 1122 ms. Trace:
I20260810 07:53:02.724898 1071 tablet_bootstrap.cc:492] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.726475 1071 ts_tablet_manager.cc:1403] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.008s
I20260810 07:53:02.726398 627 rpcz_store.cc:276] 0810 07:53:01.603097 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.610803 (+ 7706us) service_pool.cc:224] Handling call
0810 07:53:01.612582 (+ 1779us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.622671 (+ 10089us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.291302 (+668631us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.724921 (+433619us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.725729 (+ 808us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.729245 1071 raft_consensus.cc:359] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.730068 32465 catalog_manager.cc:6807] ReplaceTablet: tablet 213fa4838fe847de87091cfba97eaedc deleted and replaced by tablet f65480f06a3c4b2886aa2485f81e16f2
I20260810 07:53:02.730043 1071 raft_consensus.cc:385] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.730280 1071 raft_consensus.cc:740] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:02.732009 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 37b48c88571040f6be8d7cbd388feb58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37b48c88571040f6be8d7cbd388feb58 already in progress: creating tablet
I20260810 07:53:02.733105 1880 tablet_replica.cc:333] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:02.734176 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c8825a5b51f241a1b6539a796438bbb4" } new_table_name: "default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e" modify_external_catalogs: true
I20260810 07:53:02.735373 1071 consensus_queue.cc:260] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.733855 1880 raft_consensus.cc:2243] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:02.738137 1047 ts_tablet_manager.cc:1434] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.168s user 0.006s sys 0.001s
I20260810 07:53:02.739261 1047 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:02.741768 1880 raft_consensus.cc:2272] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:02.743721 1880 ts_tablet_manager.cc:1916] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:02.640646 1064 raft_consensus.cc:359] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.744324 1064 raft_consensus.cc:385] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.697131 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:02.745496 1071 ts_tablet_manager.cc:1434] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260810 07:53:02.744738 1064 raft_consensus.cc:740] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.745402 894 raft_consensus.cc:3060] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:02.727999 862 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:02.753705 1071 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:02.763502 635 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 210) took 1069 ms. Trace:
I20260810 07:53:02.775810 1071 tablet_bootstrap.cc:654] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.776139 1753 raft_consensus.cc:493] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:02.778017 1753 raft_consensus.cc:515] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.781339 1753 leader_election.cc:290] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:02.783437 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:02.785148 412 raft_consensus.cc:3060] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:02.786587 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:02.793361 645 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 205) took 1191 ms. Trace:
I20260810 07:53:02.793789 645 rpcz_store.cc:276] 0810 07:53:01.602258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.608869 (+ 6611us) service_pool.cc:224] Handling call
0810 07:53:01.610308 (+ 1439us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.610427 (+ 119us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.586446 (+976019us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.792911 (+206465us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.793303 (+ 392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.798614 1880 ts_tablet_manager.cc:1929] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:02.799002 1880 log.cc:1199] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/d7626d30f4574658bb226f2e915ee0de
I20260810 07:53:02.799742 1880 ts_tablet_manager.cc:1950] T d7626d30f4574658bb226f2e915ee0de P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:02.802497 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]) successfully deleted
W20260810 07:53:02.786053 792 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:02.806231 792 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:02.815773 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:02.819854 1946 raft_consensus.cc:493] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:02.820355 1946 raft_consensus.cc:515] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.822379 1946 leader_election.cc:290] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:02.831678 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:02.831923 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:02.836851 1064 consensus_queue.cc:260] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.839613 1874 ts_tablet_manager.cc:1916] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:02.788867 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 93178.5us
W20260810 07:53:02.795895 573 consensus_peers.cc:597] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:02.797886 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:02.840773 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 45010.3us
I20260810 07:53:02.799592 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c171d3358e44ae2962ab3c4c377b0f1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:02.841413 676 raft_consensus.cc:2468] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:02.846830 1071 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.847301 1071 ts_tablet_manager.cc:1403] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.094s user 0.022s sys 0.008s
I20260810 07:53:02.763973 635 rpcz_store.cc:276] 0810 07:53:01.694981 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.709142 (+ 14161us) service_pool.cc:224] Handling call
0810 07:53:01.764025 (+ 54883us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:01.764169 (+ 144us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.565491 (+801322us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.762868 (+197377us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.763416 (+ 548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.805588 623 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 212) took 1053 ms. Trace:
I20260810 07:53:02.850056 1071 raft_consensus.cc:359] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.850769 1071 raft_consensus.cc:385] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.850319 623 rpcz_store.cc:276] 0810 07:53:01.753967 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.916796 (+162829us) service_pool.cc:224] Handling call
0810 07:53:02.111849 (+195053us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:02.111972 (+ 123us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:02.642747 (+530775us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:02.801994 (+159247us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:02.805528 (+ 3534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:02.851042 1071 raft_consensus.cc:740] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.851915 1071 consensus_queue.cc:260] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.857460 1071 ts_tablet_manager.cc:1434] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260810 07:53:02.861353 1071 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:02.867303 1071 tablet_bootstrap.cc:654] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.835695 1047 tablet_bootstrap.cc:654] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.824457 32463 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:02.878724 1071 tablet_bootstrap.cc:492] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.879148 1071 ts_tablet_manager.cc:1403] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20260810 07:53:02.879041 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:02.879948 382 tablet_service.cc:1511] Processing CreateTablet for tablet 8d717f39a6704199abc34b220d795e48 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:02.877521 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c43aeafc19a240e6991b69249b0516fa consensus state has no leader
I20260810 07:53:02.881479 369 tablet_service.cc:1511] Processing CreateTablet for tablet 2c49060c69f2434db4e9f83f2147d7d1 (DEFAULT_TABLE table=default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:02.881704 1071 raft_consensus.cc:359] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.882436 1071 raft_consensus.cc:385] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.882637 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c49060c69f2434db4e9f83f2147d7d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.882696 1071 raft_consensus.cc:740] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.882038 32469 catalog_manager.cc:5719] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:02.883632 1071 consensus_queue.cc:260] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.884215 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:02.884135 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_c4a69cf1b9124eb0a73fd461f95a9c2a [id=837b105c639d460fbd573628e3778262]). Expected version 1 got 0
I20260810 07:53:02.884979 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d717f39a6704199abc34b220d795e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.885609 862 tablet_service.cc:1511] Processing CreateTablet for tablet 8d717f39a6704199abc34b220d795e48 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:02.886754 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d717f39a6704199abc34b220d795e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:02.886561 32463 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_98bd794f5968403fb88241d8e84ae242" } modify_external_catalogs: true
I20260810 07:53:02.887209 32463 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_98bd794f5968403fb88241d8e84ae242" } modify_external_catalogs: true
I20260810 07:53:02.888820 1071 ts_tablet_manager.cc:1434] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260810 07:53:02.883908 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d2f777ade76044e5a3f80922d25b9c86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:02.892490 1071 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:02.893457 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2f777ade76044e5a3f80922d25b9c86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.895162 32469 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
W20260810 07:53:02.897167 32461 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet acc89d15b4d544b7bc8abc12a4c6d3e5 consensus state has no leader
I20260810 07:53:02.897944 32461 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 223) took 2288 ms. Trace:
I20260810 07:53:02.898273 1071 tablet_bootstrap.cc:654] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:02.904995 1985 consensus_queue.cc:1048] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:02.891623 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet dd22d1029ff249a0a9e05862293261ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd22d1029ff249a0a9e05862293261ff already in progress: creating tablet
I20260810 07:53:02.898326 32461 rpcz_store.cc:276] 0810 07:53:00.609775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.616810 (+ 7035us) service_pool.cc:224] Handling call
0810 07:53:00.617696 (+ 886us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:00.623151 (+ 5455us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:00.623856 (+ 705us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:00.624075 (+ 219us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:00.906518 (+282443us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:02.897857 (+1991339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:00.870573 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:00.870988 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:00.871015 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:53:00.871030 (+ 15us) tablet.cc:662] Decoding operations
0810 07:53:00.890936 (+ 19906us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:00.891018 (+ 82us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:00.891046 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:00.891248 (+ 202us) tablet.cc:701] Row locks acquired
0810 07:53:00.891263 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:00.891399 (+ 136us) write_op.cc:270] Start()
0810 07:53:00.891514 (+ 115us) write_op.cc:276] Timestamp: P: 1786348380891316 usec, L: 0
0810 07:53:00.891529 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:53:00.892139 (+ 610us) log.cc:844] Serialized 405 byte log entry
0810 07:53:00.904768 (+ 12629us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.905171 (+ 403us) write_op.cc:301] APPLY: starting
0810 07:53:00.905241 (+ 70us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:00.905406 (+ 165us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:00.905496 (+ 90us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:00.905670 (+ 174us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:00.905811 (+ 141us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:00.905862 (+ 51us) write_op.cc:312] APPLY: finished
0810 07:53:00.906054 (+ 192us) log.cc:844] Serialized 23 byte log entry
0810 07:53:00.906201 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.906274 (+ 73us) write_op.cc:454] Released schema lock
0810 07:53:00.906306 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:00.906970 (+ 664us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.907014 (+ 44us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":222,"apply.run_cpu_time_us":1395,"apply.run_wall_time_us":1720,"num_ops":1,"prepare.queue_time_us":243313,"prepare.run_cpu_time_us":2516,"prepare.run_wall_time_us":31610,"raft.queue_time_us":452,"raft.run_cpu_time_us":1045,"raft.run_wall_time_us":4800,"replication_time_us":13116}]]}
I20260810 07:53:02.915169 1071 tablet_bootstrap.cc:492] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.915678 1071 ts_tablet_manager.cc:1403] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.006s
I20260810 07:53:02.855544 792 leader_election.cc:304] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:02.918270 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d4ab5ccfe1de44e092fb9a3183698973"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:02.918437 1071 raft_consensus.cc:359] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.919162 1071 raft_consensus.cc:385] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:02.919473 1071 raft_consensus.cc:740] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:02.919873 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4ab5ccfe1de44e092fb9a3183698973 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.920344 1071 consensus_queue.cc:260] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:02.924160 32460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 37) took 2837 ms (client timeout 1000 ms). Trace:
W20260810 07:53:02.924490 32460 rpcz_store.cc:269] 0810 07:53:00.087023 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.090513 (+ 3490us) service_pool.cc:224] Handling call
0810 07:53:02.923922 (+2833409us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":23}
I20260810 07:53:02.925971 1071 ts_tablet_manager.cc:1434] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260810 07:53:02.927799 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 38) took 1745 ms (client timeout 1000 ms). Trace:
W20260810 07:53:02.928138 32462 rpcz_store.cc:269] 0810 07:53:01.182182 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.182354 (+ 172us) service_pool.cc:224] Handling call
0810 07:53:01.205431 (+ 23077us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:02.927608 (+1722177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14971,"reported_tablets":23,"spinlock_wait_cycles":43084160}
I20260810 07:53:02.930248 1071 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:02.930552 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_90bfccd382de4eebb7f693185dab51d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:02.932034 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90bfccd382de4eebb7f693185dab51d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:02.932478 1047 tablet_bootstrap.cc:492] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:02.933097 32460 master_service.cc:960] ReplaceTablet: received request to replace tablet a89cdc44a8c64ccd83fd30855798f87b from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:02.936391 1071 tablet_bootstrap.cc:654] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:02.938679 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 132484us
I20260810 07:53:02.941596 1047 ts_tablet_manager.cc:1403] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.203s user 0.013s sys 0.001s
I20260810 07:53:02.947580 1071 tablet_bootstrap.cc:492] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:02.947978 1071 ts_tablet_manager.cc:1403] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
I20260810 07:53:02.949342 32464 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279] alter complete (version 1)
I20260810 07:53:02.939260 1926 raft_consensus.cc:2749] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:02.947743 1047 raft_consensus.cc:359] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:02.959095 308 leader_election.cc:343] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
I20260810 07:53:02.959571 308 leader_election.cc:304] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:02.959957 1047 raft_consensus.cc:385] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:53:02.960217 308 raft_consensus.cc:2686] T 338e9b0e89d44295973d4003abd36649 P 60781ef3c8b84a8788f43d77e1bb3b19: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260810 07:53:02.960223 1047 raft_consensus.cc:740] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:02.970423 1047 consensus_queue.cc:260] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:02.941726 1064 ts_tablet_manager.cc:1434] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.344s user 0.008s sys 0.000s
W20260810 07:53:02.913053 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 213fa4838fe847de87091cfba97eaedc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 213fa4838fe847de87091cfba97eaedc already in progress: creating tablet
W20260810 07:53:02.973865 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 60829us
W20260810 07:53:02.990933 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 26) took 3319 ms (client timeout 1000 ms). Trace:
I20260810 07:53:02.992559 1064 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:03.016031 1047 ts_tablet_manager.cc:1434] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.071s user 0.003s sys 0.002s
I20260810 07:53:03.018208 864 tablet_service.cc:1511] Processing CreateTablet for tablet 2c49060c69f2434db4e9f83f2147d7d1 (DEFAULT_TABLE table=default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.019992 852 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (DEFAULT_TABLE table=default.Table_4c5928f06794426aa64b4c1a6111d2bf [id=616669a2db4f43d9b352efabdfc7b015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.021363 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c49060c69f2434db4e9f83f2147d7d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.021569 1047 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:03.025282 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe71e69dfd74a3b826050bd7edc6b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.029042 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.029366 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.030579 635 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.031566 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.033119 635 tablet_service.cc:1511] Processing CreateTablet for tablet 8d717f39a6704199abc34b220d795e48 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.034353 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d717f39a6704199abc34b220d795e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.035954 851 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6df1791e5146a38f3cf0304b5034cb (DEFAULT_TABLE table=default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.037187 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6df1791e5146a38f3cf0304b5034cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.042568 894 raft_consensus.cc:2468] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:03.049757 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.051678 633 tablet_service.cc:1511] Processing CreateTablet for tablet 2c49060c69f2434db4e9f83f2147d7d1 (DEFAULT_TABLE table=default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.056108 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c49060c69f2434db4e9f83f2147d7d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.054968 1047 tablet_bootstrap.cc:654] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:03.083986 2092 tablet_replica.cc:333] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:03.084074 32463 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 3f63757d38b44a4aa991ac207f8060cb
I20260810 07:53:03.085141 2092 raft_consensus.cc:2243] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:03.086436 2092 raft_consensus.cc:2272] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:03.092325 32464 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1] alter complete (version 4)
I20260810 07:53:03.099692 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.102999 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.106343 851 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.108886 851 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:02.991555 32470 rpcz_store.cc:269] 0810 07:52:59.671391 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:59.715098 (+ 43707us) service_pool.cc:224] Handling call
0810 07:53:02.990689 (+3275591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:00.474773 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:00.475263 (+ 490us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:00.475296 (+ 33us) write_op.cc:435] Acquired schema lock
0810 07:53:00.475315 (+ 19us) tablet.cc:662] Decoding operations
0810 07:53:00.475703 (+ 388us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:00.475757 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:00.475791 (+ 34us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:00.476134 (+ 343us) tablet.cc:701] Row locks acquired
0810 07:53:00.476154 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:00.476280 (+ 126us) write_op.cc:270] Start()
0810 07:53:00.476379 (+ 99us) write_op.cc:276] Timestamp: P: 1786348380476187 usec, L: 0
0810 07:53:00.476579 (+ 200us) op_driver.cc:348] REPLICATION: starting
0810 07:53:00.477080 (+ 501us) log.cc:844] Serialized 913 byte log entry
0810 07:53:00.500644 (+ 23564us) op_driver.cc:464] REPLICATION: finished
0810 07:53:00.503129 (+ 2485us) write_op.cc:301] APPLY: starting
0810 07:53:00.503181 (+ 52us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:00.503277 (+ 96us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:00.503361 (+ 84us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:00.503649 (+ 288us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:00.504158 (+ 509us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:00.504235 (+ 77us) write_op.cc:312] APPLY: finished
0810 07:53:00.504463 (+ 228us) log.cc:844] Serialized 29 byte log entry
0810 07:53:00.504610 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:00.504696 (+ 86us) write_op.cc:454] Released schema lock
0810 07:53:00.504764 (+ 68us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":48526,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":2362,"apply.run_cpu_time_us":1648,"apply.run_wall_time_us":1844,"mutex_wait_us":26052,"num_ops":2,"prepare.queue_time_us":26726,"prepare.run_cpu_time_us":2490,"prepare.run_wall_time_us":6248,"raft.queue_time_us":2532,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":15,"replication_time_us":23980,"thread_start_us":87275,"threads_started":1}]]}
I20260810 07:53:03.124306 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.128791 1047 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:03.130504 851 tablet_service.cc:1511] Processing CreateTablet for tablet cc9959255b4d46e29b9164027638e38e (DEFAULT_TABLE table=default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.129216 1047 ts_tablet_manager.cc:1403] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.108s user 0.010s sys 0.000s
I20260810 07:53:03.134137 860 tablet_service.cc:1511] Processing CreateTablet for tablet 6cd9484084294acfba9613fa49f369b2 (DEFAULT_TABLE table=default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.137290 574 leader_election.cc:304] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:03.139015 1671 raft_consensus.cc:2804] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:03.140226 1671 raft_consensus.cc:697] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:03.141810 1671 consensus_queue.cc:237] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.147125 1047 raft_consensus.cc:359] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.152634 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_10b86803e3984d76bdb6f83b57a9451f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:03.154282 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10b86803e3984d76bdb6f83b57a9451f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:03.157737 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9959255b4d46e29b9164027638e38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.158360 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cd9484084294acfba9613fa49f369b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.171043 1047 raft_consensus.cc:385] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.172060 1047 raft_consensus.cc:740] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.174464 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.177342 1047 consensus_queue.cc:260] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.178452 849 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.187060 1047 ts_tablet_manager.cc:1434] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
W20260810 07:53:03.190557 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 39) took 976 ms (client timeout 1000 ms). Trace:
I20260810 07:53:03.182861 1946 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.191087 1946 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.195995 382 tablet_service.cc:1558] Processing DeleteTablet for tablet b2bfffdd01524eada0a85c1a525325a0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c13f622081847f88082489de32d58df) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.197574 382 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.198135 376 tablet_service.cc:1558] Processing DeleteTablet for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 with delete_type TABLET_DATA_DELETED (replaced by tablet a89cdc44a8c64ccd83fd30855798f87b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.198364 2088 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.198890 2088 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.201218 1946 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.203593 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:03.204211 896 raft_consensus.cc:2468] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:03.206002 2088 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:03.207250 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 248192us
I20260810 07:53:03.194430 412 raft_consensus.cc:3060] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.210467 415 raft_consensus.cc:3060] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.216420 308 leader_election.cc:304] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:03.216455 407 raft_consensus.cc:3060] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.218084 308 leader_election.cc:304] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:03.220094 1985 consensus_queue.cc:1048] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:03.221652 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 338e9b0e89d44295973d4003abd36649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:52 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:03.230114 32467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 24) took 2921 ms (client timeout 1000 ms). Trace:
I20260810 07:53:03.232430 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:03.234371 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 25) took 1822 ms (client timeout 1000 ms). Trace:
I20260810 07:53:03.235348 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.237123 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.237264 1064 tablet_bootstrap.cc:654] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:03.238088 1071 raft_consensus.cc:359] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.239790 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.240445 1071 raft_consensus.cc:385] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.241330 1880 tablet_replica.cc:333] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:03.242270 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 09712cfebab34e56ba844e7aa0e8d152 with delete_type TABLET_DATA_DELETED (replaced by tablet e36780fb709f427f872f3e58df7b7e81) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.243197 1071 raft_consensus.cc:740] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.243691 1047 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:03.243927 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.243942 1071 consensus_queue.cc:260] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.244482 676 raft_consensus.cc:2468] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:03.234584 309 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:03.253165 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ec00a494816409684205c7f4435a079 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:50 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.258262 2092 ts_tablet_manager.cc:1916] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:03.262837 864 tablet_service.cc:1511] Processing CreateTablet for tablet c6080c8b12014fe09d48e307d394f9c9 (DEFAULT_TABLE table=default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.264266 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6080c8b12014fe09d48e307d394f9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.270120 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.271941 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 92e2678ab165486dbbb69c7c183a7fa4 with delete_type TABLET_DATA_DELETED (replaced by tablet dd22d1029ff249a0a9e05862293261ff) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.275409 376 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.280359 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_4ec9847b17c44e009c6162f1417d5a13" } new_table_name: "default.Table_ae927dcca7ec42b99eee31143de3bf0b" modify_external_catalogs: true
I20260810 07:53:03.285897 32461 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3cec5176ba3946aebc2391756561ecc1" } modify_external_catalogs: true
I20260810 07:53:03.286815 32461 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_3cec5176ba3946aebc2391756561ecc1" } modify_external_catalogs: true
I20260810 07:53:03.296885 1880 raft_consensus.cc:2243] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:03.297458 1880 raft_consensus.cc:2272] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:03.299417 1880 ts_tablet_manager.cc:1916] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:03.234745 32468 rpcz_store.cc:269] 0810 07:53:01.411656 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.414801 (+ 3145us) service_pool.cc:224] Handling call
0810 07:53:03.234092 (+1819291us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:02.316276 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:02.316642 (+ 366us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:02.316663 (+ 21us) write_op.cc:435] Acquired schema lock
0810 07:53:02.316680 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:02.316952 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:02.316991 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:02.317024 (+ 33us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:02.317232 (+ 208us) tablet.cc:701] Row locks acquired
0810 07:53:02.317251 (+ 19us) write_op.cc:260] PREPARE: finished
0810 07:53:02.317382 (+ 131us) write_op.cc:270] Start()
0810 07:53:02.317480 (+ 98us) write_op.cc:276] Timestamp: P: 1786348382317333 usec, L: 0
0810 07:53:02.317502 (+ 22us) op_driver.cc:348] REPLICATION: starting
0810 07:53:02.318079 (+ 577us) log.cc:844] Serialized 533 byte log entry
0810 07:53:02.329864 (+ 11785us) op_driver.cc:464] REPLICATION: finished
0810 07:53:02.332365 (+ 2501us) write_op.cc:301] APPLY: starting
0810 07:53:02.332426 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:02.332561 (+ 135us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:02.332661 (+ 100us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:02.332874 (+ 213us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:02.333022 (+ 148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:02.333073 (+ 51us) write_op.cc:312] APPLY: finished
0810 07:53:02.333260 (+ 187us) log.cc:844] Serialized 23 byte log entry
0810 07:53:02.333407 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:02.333478 (+ 71us) write_op.cc:454] Released schema lock
0810 07:53:02.333514 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:02.333730 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:02.333753 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":114565,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":2339,"apply.run_cpu_time_us":1350,"apply.run_wall_time_us":1351,"num_ops":1,"prepare.queue_time_us":138,"prepare.run_cpu_time_us":2015,"prepare.run_wall_time_us":2014,"replication_time_us":12265}]]}
I20260810 07:53:03.312969 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_1899e690e1fd4821a2930f385431fafb" } new_table_name: "default.Table_332f1a619c404512a115b8f6cb52402f" modify_external_catalogs: true
I20260810 07:53:03.316205 2103 raft_consensus.cc:2804] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:03.316785 2103 raft_consensus.cc:493] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:03.317255 2103 raft_consensus.cc:3060] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:03.230619 32467 rpcz_store.cc:269] 0810 07:53:00.308983 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:00.309115 (+ 132us) service_pool.cc:224] Handling call
0810 07:53:03.229799 (+2920684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:01.459482 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:01.459936 (+ 454us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:01.459960 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:53:01.459971 (+ 11us) tablet.cc:662] Decoding operations
0810 07:53:01.460262 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:01.460312 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:01.460334 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:01.460519 (+ 185us) tablet.cc:701] Row locks acquired
0810 07:53:01.460542 (+ 23us) write_op.cc:260] PREPARE: finished
0810 07:53:01.460669 (+ 127us) write_op.cc:270] Start()
0810 07:53:01.460773 (+ 104us) write_op.cc:276] Timestamp: P: 1786348381460599 usec, L: 0
0810 07:53:01.460788 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:53:01.461315 (+ 527us) log.cc:844] Serialized 533 byte log entry
0810 07:53:01.494782 (+ 33467us) op_driver.cc:464] REPLICATION: finished
0810 07:53:01.495218 (+ 436us) write_op.cc:301] APPLY: starting
0810 07:53:01.495275 (+ 57us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:01.495407 (+ 132us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:01.495500 (+ 93us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:01.495690 (+ 190us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:01.495821 (+ 131us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:01.495864 (+ 43us) write_op.cc:312] APPLY: finished
0810 07:53:01.496028 (+ 164us) log.cc:844] Serialized 23 byte log entry
0810 07:53:01.496190 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:01.496251 (+ 61us) write_op.cc:454] Released schema lock
0810 07:53:01.496281 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:01.496521 (+ 240us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:01.496545 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":50319,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":273,"apply.run_cpu_time_us":1275,"apply.run_wall_time_us":1275,"num_ops":1,"prepare.queue_time_us":3282,"prepare.run_cpu_time_us":2152,"prepare.run_wall_time_us":2347,"raft.queue_time_us":90,"raft.run_cpu_time_us":26,"raft.run_wall_time_us":26,"replication_time_us":33880}]]}
I20260810 07:53:03.323436 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_712d9cfefe684238a26898997e816b64" } modify_external_catalogs: true
I20260810 07:53:03.324116 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_712d9cfefe684238a26898997e816b64" } modify_external_catalogs: true
W20260810 07:53:03.332111 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 143438us
I20260810 07:53:03.353657 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.354416 676 raft_consensus.cc:2468] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
W20260810 07:53:03.191180 32464 rpcz_store.cc:269] 0810 07:53:02.214268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.240031 (+ 25763us) service_pool.cc:224] Handling call
0810 07:53:02.582916 (+342885us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:02.582967 (+ 51us) write_op.cc:454] Released schema lock
0810 07:53:03.190488 (+607521us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:02.560911 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:02.561299 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:02.561325 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:02.561337 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:02.561597 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:02.561637 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:02.561659 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:02.561809 (+ 150us) tablet.cc:701] Row locks acquired
0810 07:53:02.561823 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:02.561956 (+ 133us) write_op.cc:270] Start()
0810 07:53:02.562037 (+ 81us) write_op.cc:276] Timestamp: P: 1786348382561891 usec, L: 0
0810 07:53:02.562053 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:02.562535 (+ 482us) log.cc:844] Serialized 356 byte log entry
0810 07:53:02.573387 (+ 10852us) op_driver.cc:464] REPLICATION: finished
0810 07:53:02.573866 (+ 479us) write_op.cc:301] APPLY: starting
0810 07:53:02.573927 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:02.574054 (+ 127us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:02.574266 (+ 212us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:02.574448 (+ 182us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:02.574619 (+ 171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:02.574680 (+ 61us) write_op.cc:312] APPLY: finished
0810 07:53:02.574913 (+ 233us) log.cc:844] Serialized 23 byte log entry
0810 07:53:02.575090 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:02.575174 (+ 84us) write_op.cc:454] Released schema lock
0810 07:53:02.575215 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:53:03.026645 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:03.027012 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:03.027035 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:03.027047 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:03.027293 (+ 246us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:03.027328 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:03.027357 (+ 29us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:03.027503 (+ 146us) tablet.cc:701] Row locks acquired
0810 07:53:03.027515 (+ 12us) write_op.cc:260] PREPARE: finished
0810 07:53:03.027625 (+ 110us) write_op.cc:270] Start()
0810 07:53:03.027710 (+ 85us) write_op.cc:276] Timestamp: P: 1786348383027546 usec, L: 0
0810 07:53:03.027724 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:03.028170 (+ 446us) log.cc:844] Serialized 356 byte log entry
0810 07:53:03.064084 (+ 35914us) op_driver.cc:464] REPLICATION: finished
0810 07:53:03.081883 (+ 17799us) write_op.cc:301] APPLY: starting
0810 07:53:03.081934 (+ 51us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:03.082017 (+ 83us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:03.082115 (+ 98us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:03.082328 (+ 213us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:03.082458 (+ 130us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:03.082500 (+ 42us) write_op.cc:312] APPLY: finished
0810 07:53:03.082675 (+ 175us) log.cc:844] Serialized 23 byte log entry
0810 07:53:03.082858 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:03.082943 (+ 85us) write_op.cc:454] Released schema lock
0810 07:53:03.082993 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:03.140853 (+ 57860us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:03.140878 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15002,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":325,"apply.run_cpu_time_us":1563,"apply.run_wall_time_us":1562,"num_ops":1,"prepare.queue_time_us":1166,"prepare.run_cpu_time_us":1936,"prepare.run_wall_time_us":1936,"raft.queue_time_us":4420,"raft.run_cpu_time_us":876,"raft.run_wall_time_us":1039,"replication_time_us":11224}],["op",{"apply.queue_time_us":17677,"apply.run_cpu_time_us":1371,"apply.run_wall_time_us":1371,"num_ops":1,"prepare.queue_time_us":53309,"prepare.run_cpu_time_us":5952,"prepare.run_wall_time_us":114167,"raft.queue_time_us":23262,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":7,"replication_time_us":36294,"thread_start_us":137669,"threads_started":2}]]}
I20260810 07:53:03.358451 862 tablet_service.cc:1511] Processing CreateTablet for tablet 9350b151227d45e6bae8adddd8fff86b (DEFAULT_TABLE table=default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.361754 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_94e83c4488294b578216286d2a51ac73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:03.363401 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94e83c4488294b578216286d2a51ac73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:03.337091 1969 raft_consensus.cc:493] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.367719 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]). Expected version 2 got 0
I20260810 07:53:03.376017 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9350b151227d45e6bae8adddd8fff86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.380820 1047 tablet_bootstrap.cc:654] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:03.353219 1971 consensus_queue.cc:1048] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260810 07:53:03.385600 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ae4a28e89714c65b7551f36d1dc78b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:57 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.371017 1969 raft_consensus.cc:515] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.389057 1969 leader_election.cc:290] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.374517 852 tablet_service.cc:1511] Processing CreateTablet for tablet f4a46a037cae4f599398b289bb6a2d84 (DEFAULT_TABLE table=default.Table_d4c51e03eea74f81a39e48e631649307 [id=466e3662813242f9b2c85763c28ae42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.391976 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a46a037cae4f599398b289bb6a2d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.394925 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.323119 2114 raft_consensus.cc:2804] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:03.396891 2114 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:03.397239 2114 raft_consensus.cc:3060] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.352685 369 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (DEFAULT_TABLE table=default.Table_4c5928f06794426aa64b4c1a6111d2bf [id=616669a2db4f43d9b352efabdfc7b015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.354833 376 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6df1791e5146a38f3cf0304b5034cb (DEFAULT_TABLE table=default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.398651 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6df1791e5146a38f3cf0304b5034cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.399286 635 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.400502 635 tablet_service.cc:1558] Processing DeleteTablet for tablet cf45a33485e9463d89b5ce8affa0bd22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:54 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.405030 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.406219 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 08520a2ac0f64c3abd534a32100839a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a61dbafb4f7444e690cea0d003d38f22) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.407220 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe71e69dfd74a3b826050bd7edc6b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.407783 635 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (DEFAULT_TABLE table=default.Table_4c5928f06794426aa64b4c1a6111d2bf [id=616669a2db4f43d9b352efabdfc7b015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.408990 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe71e69dfd74a3b826050bd7edc6b6a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:03.365104 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 147144us
W20260810 07:53:03.365599 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 133772us
I20260810 07:53:03.422367 308 leader_election.cc:304] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:03.375775 2109 raft_consensus.cc:493] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.423125 2109 raft_consensus.cc:515] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.426493 2116 raft_consensus.cc:2749] T e38ab80cd5c441f3ba20bf00577b223c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:03.426942 633 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6df1791e5146a38f3cf0304b5034cb (DEFAULT_TABLE table=default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.428308 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6df1791e5146a38f3cf0304b5034cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.431732 860 tablet_service.cc:1511] Processing CreateTablet for tablet a4863f3f03104d31ad4dafda6f33c9af (DEFAULT_TABLE table=default.Table_712d9cfefe684238a26898997e816b64 [id=8753c36aa07540b68965b2a61a467bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.432996 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4863f3f03104d31ad4dafda6f33c9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.434340 1926 raft_consensus.cc:493] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.434793 1926 raft_consensus.cc:515] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.439844 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:03.377413 1071 ts_tablet_manager.cc:1434] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.429s user 0.041s sys 0.092s
I20260810 07:53:03.383430 382 tablet_service.cc:1511] Processing CreateTablet for tablet cc9959255b4d46e29b9164027638e38e (DEFAULT_TABLE table=default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.440877 1926 leader_election.cc:290] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:03.441236 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9959255b4d46e29b9164027638e38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.443173 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_28f4942b995147189f6334d58a30f047"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:03.383458 367 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.384967 375 tablet_service.cc:1511] Processing CreateTablet for tablet 6cd9484084294acfba9613fa49f369b2 (DEFAULT_TABLE table=default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.445679 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cd9484084294acfba9613fa49f369b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.447479 623 tablet_service.cc:1511] Processing CreateTablet for tablet cc9959255b4d46e29b9164027638e38e (DEFAULT_TABLE table=default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.447573 645 tablet_service.cc:1511] Processing CreateTablet for tablet 6cd9484084294acfba9613fa49f369b2 (DEFAULT_TABLE table=default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.386775 385 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.448807 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9959255b4d46e29b9164027638e38e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.388335 371 tablet_service.cc:1511] Processing CreateTablet for tablet c6080c8b12014fe09d48e307d394f9c9 (DEFAULT_TABLE table=default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.449649 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6080c8b12014fe09d48e307d394f9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.449561 622 tablet_service.cc:1511] Processing CreateTablet for tablet c6080c8b12014fe09d48e307d394f9c9 (DEFAULT_TABLE table=default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.449843 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.451109 620 tablet_service.cc:1511] Processing CreateTablet for tablet 9350b151227d45e6bae8adddd8fff86b (DEFAULT_TABLE table=default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.390343 380 tablet_service.cc:1511] Processing CreateTablet for tablet 9350b151227d45e6bae8adddd8fff86b (DEFAULT_TABLE table=default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.453357 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:03.454823 32460 catalog_manager.cc:6807] ReplaceTablet: tablet a89cdc44a8c64ccd83fd30855798f87b deleted and replaced by tablet aed14fa26b9a46a4b3e9a43bac05a795
I20260810 07:53:03.457398 849 tablet_service.cc:1558] Processing DeleteTablet for tablet a89cdc44a8c64ccd83fd30855798f87b with delete_type TABLET_DATA_DELETED (replaced by tablet aed14fa26b9a46a4b3e9a43bac05a795) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.452628 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9350b151227d45e6bae8adddd8fff86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.460603 1064 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:03.461300 1064 ts_tablet_manager.cc:1403] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.469s user 0.008s sys 0.004s
W20260810 07:53:03.461349 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c43aeafc19a240e6991b69249b0516fa (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c43aeafc19a240e6991b69249b0516fa consensus state has no leader
W20260810 07:53:03.444702 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28f4942b995147189f6334d58a30f047 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:03.469838 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:03.472282 1071 ts_tablet_manager.cc:1467] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Tablet startup took 524ms
W20260810 07:53:03.472627 1071 ts_tablet_manager.cc:1469] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Trace:
0810 07:52:55.949988 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:52:55.950612 (+ 624us) service_pool.cc:224] Handling call
0810 07:52:55.952054 (+ 1442us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:52:55.952145 (+ 91us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:52:55.963256 (+ 11111us) tablet_metadata.cc:665] Metadata flushed
0810 07:52:56.159141 (+195885us) tablet_replica.cc:206] Creating consensus instance
0810 07:52:56.159565 (+ 424us) inbound_call.cc:177] Queueing success response
0810 07:53:02.929942 (+6770377us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0810 07:53:02.948229 (+ 18287us) ts_tablet_manager.cc:1435] Starting tablet replica
0810 07:53:02.948767 (+ 538us) tablet_replica.cc:261] Starting instrumentation
0810 07:53:03.194905 (+246138us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":77440,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":285,"tablet-open.queue_time_us":6770449,"thread_start_us":131447,"threads_started":1}
I20260810 07:53:03.481196 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b48c88571040f6be8d7cbd388feb58" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:03.482234 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6080c8b12014fe09d48e307d394f9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.484881 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.485673 1071 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:03.483557 2109 leader_election.cc:290] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:03.492847 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 154055us
I20260810 07:53:03.493851 1946 raft_consensus.cc:2749] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:03.496862 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:03.497426 414 raft_consensus.cc:2468] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:03.498288 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cd9484084294acfba9613fa49f369b2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:03.502297 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 27) took 1157 ms (client timeout 1000 ms). Trace:
I20260810 07:53:03.507298 2042 tablet.cc:1724] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:03.514189 620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9350b151227d45e6bae8adddd8fff86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.528756 573 leader_election.cc:304] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:03.529874 1600 raft_consensus.cc:2804] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:03.530218 1600 raft_consensus.cc:493] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:03.530512 1600 raft_consensus.cc:3060] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.531749 1874 ts_tablet_manager.cc:1929] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:03.532255 1874 log.cc:1199] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:03.533128 1874 ts_tablet_manager.cc:1950] T dd22d1029ff249a0a9e05862293261ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:03.538353 1600 raft_consensus.cc:515] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.516491 32463 catalog_manager.cc:5719] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:03.540560 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:03.541652 1880 ts_tablet_manager.cc:1929] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:03.542047 1880 log.cc:1199] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:03.541541 2106 tablet.cc:1724] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:03.542865 1880 ts_tablet_manager.cc:1950] T 449359c8e9fa4756ba149f3dce6e0ce7 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:03.547684 1600 leader_election.cc:290] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.560932 849 tablet_service.cc:1511] Processing CreateTablet for tablet a4d1c8d270f744ee9849475c9128414b (DEFAULT_TABLE table=default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.562237 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:03.562870 415 raft_consensus.cc:3060] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.574709 2103 raft_consensus.cc:515] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.586812 1047 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:03.586866 859 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb896c8922e412c97e2cc4891298830 (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.587280 1047 ts_tablet_manager.cc:1403] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.344s user 0.018s sys 0.035s
I20260810 07:53:03.589215 415 raft_consensus.cc:2468] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:03.593828 308 leader_election.cc:304] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:03.595396 2109 raft_consensus.cc:2749] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:03.595543 573 leader_election.cc:304] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:03.598388 1969 raft_consensus.cc:2804] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:03.599421 2114 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.605079 407 raft_consensus.cc:2393] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:03.599216 1969 raft_consensus.cc:697] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:03.608428 1969 consensus_queue.cc:237] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:03.572302 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 53152.3us
I20260810 07:53:03.616571 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:03.620230 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:03.621845 2103 leader_election.cc:290] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.623406 2114 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.624953 1071 tablet_bootstrap.cc:654] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:03.628602 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d5ab03facf8477eada5d1ac3ec7f5ef consensus state has no leader
W20260810 07:53:03.629485 792 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:03.629710 1064 raft_consensus.cc:359] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.630353 1064 raft_consensus.cc:385] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.630643 1064 raft_consensus.cc:740] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.631280 1064 consensus_queue.cc:260] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:03.524861 32469 rpcz_store.cc:269] 0810 07:53:02.345285 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.366451 (+ 21166us) service_pool.cc:224] Handling call
0810 07:53:03.502205 (+1135754us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:03.141160 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:03.141530 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:03.141553 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:03.141566 (+ 13us) tablet.cc:662] Decoding operations
0810 07:53:03.141808 (+ 242us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:03.141843 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:03.141865 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:03.142008 (+ 143us) tablet.cc:701] Row locks acquired
0810 07:53:03.142021 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:03.142125 (+ 104us) write_op.cc:270] Start()
0810 07:53:03.142203 (+ 78us) write_op.cc:276] Timestamp: P: 1786348383142043 usec, L: 0
0810 07:53:03.142217 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:03.142697 (+ 480us) log.cc:844] Serialized 533 byte log entry
0810 07:53:03.154404 (+ 11707us) op_driver.cc:464] REPLICATION: finished
0810 07:53:03.206586 (+ 52182us) write_op.cc:301] APPLY: starting
0810 07:53:03.206649 (+ 63us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:03.206737 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:03.206909 (+ 172us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:03.207147 (+ 238us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:03.207319 (+ 172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:03.207388 (+ 69us) write_op.cc:312] APPLY: finished
0810 07:53:03.207630 (+ 242us) log.cc:844] Serialized 23 byte log entry
0810 07:53:03.207810 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:03.207883 (+ 73us) write_op.cc:454] Released schema lock
0810 07:53:03.207920 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:03.208272 (+ 352us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:03.208310 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":59911,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":51951,"apply.run_cpu_time_us":1591,"apply.run_wall_time_us":1602,"num_ops":1,"prepare.queue_time_us":158582,"prepare.run_cpu_time_us":1921,"prepare.run_wall_time_us":53066,"raft.queue_time_us":107,"raft.run_cpu_time_us":835,"raft.run_wall_time_us":835,"replication_time_us":12018}]]}
W20260810 07:53:03.596228 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 7000d25f3bfe4c99be54a0f4789cc96c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7000d25f3bfe4c99be54a0f4789cc96c already in progress: creating tablet
W20260810 07:53:03.635861 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet b2bfffdd01524eada0a85c1a525325a0 because the tablet was not found. No further retry: Not found: Tablet not found: b2bfffdd01524eada0a85c1a525325a0
W20260810 07:53:03.636603 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because the tablet was not found. No further retry: Not found: Tablet not found: cf45a33485e9463d89b5ce8affa0bd22
W20260810 07:53:03.637326 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet f9d3fcf79a104c47a6fd10fc34f7ce08 because the tablet was not found. No further retry: Not found: Tablet not found: f9d3fcf79a104c47a6fd10fc34f7ce08
W20260810 07:53:03.638123 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 338e9b0e89d44295973d4003abd36649 because the tablet was not found. No further retry: Not found: Tablet not found: 338e9b0e89d44295973d4003abd36649
W20260810 07:53:03.638770 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 9ae4a28e89714c65b7551f36d1dc78b4 because the tablet was not found. No further retry: Not found: Tablet not found: 9ae4a28e89714c65b7551f36d1dc78b4
I20260810 07:53:03.629236 855 tablet_service.cc:1511] Processing CreateTablet for tablet e1fa9200140a41dfbd886442ae336988 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.643074 1880 tablet_replica.cc:333] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:03.643893 1880 raft_consensus.cc:2243] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:03.644410 1880 raft_consensus.cc:2272] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:03.647594 1880 ts_tablet_manager.cc:1916] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:03.653501 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:03.655048 866 tablet_service.cc:1511] Processing CreateTablet for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d (DEFAULT_TABLE table=default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.666726 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c6abf897fb4e41ad88d667a52e915abf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:03.667253 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fa9200140a41dfbd886442ae336988. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:03.668592 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6abf897fb4e41ad88d667a52e915abf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:03.671377 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d1c8d270f744ee9849475c9128414b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.672130 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb896c8922e412c97e2cc4891298830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.672785 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:03.683619 851 consensus_meta.cc:287] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.516s user 0.005s sys 0.004s
I20260810 07:53:03.689324 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 188) took 2135 ms. Trace:
I20260810 07:53:03.689881 1874 rpcz_store.cc:276] 0810 07:53:01.554076 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:01.554215 (+ 139us) service_pool.cc:224] Handling call
0810 07:53:02.633038 (+1078823us) ts_tablet_manager.cc:1151] Deleting tablet dd22d1029ff249a0a9e05862293261ff
0810 07:53:02.633098 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:03.152943 (+519845us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:03.531607 (+378664us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:03.689252 (+157645us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078231}
I20260810 07:53:03.692329 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet dd22d1029ff249a0a9e05862293261ff (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]) successfully deleted
I20260810 07:53:03.697405 1064 ts_tablet_manager.cc:1434] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.236s user 0.007s sys 0.000s
I20260810 07:53:03.698534 1064 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:03.699949 32462 master_service.cc:960] ReplaceTablet: received request to replace tablet acc89d15b4d544b7bc8abc12a4c6d3e5 from {username='slave'} at 127.0.0.1:43806
W20260810 07:53:03.704988 32468 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f6b54db926fa4032b6408d3d309de35b consensus state has no leader
I20260810 07:53:03.707909 851 tablet_service.cc:1511] Processing CreateTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 (DEFAULT_TABLE table=default.Table_eb0339f23b7749ccb8c8081ee9dcae9d [id=caa8fc2fafc14076aaf44b9056c04470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.709080 851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fa93c49cb54e5fa03034eda6fc2300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.714177 1047 raft_consensus.cc:359] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.714843 1047 raft_consensus.cc:385] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.715152 1047 raft_consensus.cc:740] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.724515 1874 tablet_replica.cc:333] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:03.727162 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c171d3358e44ae2962ab3c4c377b0f1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:03.727939 676 raft_consensus.cc:3060] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.728658 1874 raft_consensus.cc:2243] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:03.729388 1874 raft_consensus.cc:2272] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:03.731050 1874 ts_tablet_manager.cc:1916] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:03.733832 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:03.734403 678 raft_consensus.cc:3060] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.740760 860 tablet_service.cc:1511] Processing CreateTablet for tablet 6db16797cabc48fe8ec82355ce3f16be (DEFAULT_TABLE table=default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.745082 676 raft_consensus.cc:2468] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:03.745055 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 114891us
I20260810 07:53:03.746515 792 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:03.747586 2088 raft_consensus.cc:2804] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:03.747952 2088 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:03.748261 2088 raft_consensus.cc:3060] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.755623 1071 tablet_bootstrap.cc:492] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:03.756124 1071 ts_tablet_manager.cc:1403] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.271s user 0.013s sys 0.004s
I20260810 07:53:03.758812 1071 raft_consensus.cc:359] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.759543 1071 raft_consensus.cc:385] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.759841 1071 raft_consensus.cc:740] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.767659 862 tablet_service.cc:1511] Processing CreateTablet for tablet 849197942f134542a387a10167fc7481 (DEFAULT_TABLE table=default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.768986 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_1256df14cdbb41e7b60d872ca8670702"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:03.770762 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1256df14cdbb41e7b60d872ca8670702 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:03.771477 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:53:03.772819 309 leader_election.cc:343] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
W20260810 07:53:03.779716 792 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:03.760834 1071 consensus_queue.cc:260] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.785228 32461 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet fe8b4892a53742f48dd43397c4f8e88f
I20260810 07:53:03.750797 678 raft_consensus.cc:2468] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:03.791178 1071 ts_tablet_manager.cc:1434] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.035s user 0.007s sys 0.000s
I20260810 07:53:03.792145 1071 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:03.793269 308 leader_election.cc:304] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:03.716185 1047 consensus_queue.cc:260] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.794637 2114 raft_consensus.cc:2804] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:03.795254 2114 raft_consensus.cc:697] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:03.796228 1047 ts_tablet_manager.cc:1434] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.209s user 0.008s sys 0.000s
I20260810 07:53:03.796331 2114 consensus_queue.cc:237] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.799314 1969 raft_consensus.cc:493] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.799909 1969 raft_consensus.cc:515] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:03.775319 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 179848us
I20260810 07:53:03.777467 2092 ts_tablet_manager.cc:1929] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:03.803714 2092 log.cc:1199] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/5e4b3ca46c9c44559761e25bcde3e927
I20260810 07:53:03.805064 2092 ts_tablet_manager.cc:1950] T 5e4b3ca46c9c44559761e25bcde3e927 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:03.824481 1047 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:03.830205 308 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:03.640535 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 449359c8e9fa4756ba149f3dce6e0ce7 (table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]) successfully deleted
W20260810 07:53:03.836251 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 195707us
W20260810 07:53:03.837059 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 5e4b3ca46c9c44559761e25bcde3e927 because the tablet was not found. No further retry: Not found: Tablet not found: 5e4b3ca46c9c44559761e25bcde3e927
I20260810 07:53:03.852979 2114 raft_consensus.cc:2804] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:03.853636 2114 raft_consensus.cc:697] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:03.857126 2114 consensus_queue.cc:237] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.869463 1880 ts_tablet_manager.cc:1929] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:03.869926 1880 log.cc:1199] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:03.833665 2092 tablet_replica.cc:333] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:03.870765 1880 ts_tablet_manager.cc:1950] T c1faed468bfa4d1ba5d92c0b9b194781 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:03.871240 2092 raft_consensus.cc:2243] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:03.875916 2092 raft_consensus.cc:2272] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:03.877488 1071 tablet_bootstrap.cc:654] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:03.880586 2092 ts_tablet_manager.cc:1916] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:03.881070 2103 raft_consensus.cc:493] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.881764 2103 raft_consensus.cc:515] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.885282 2103 leader_election.cc:290] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.889235 1880 tablet_replica.cc:333] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:03.893762 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:03.894644 1880 raft_consensus.cc:2243] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:03.895416 1880 pending_rounds.cc:70] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Trying to abort 1 pending ops.
I20260810 07:53:03.895685 1880 pending_rounds.cc:77] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882980042715136 op_type: NO_OP noop_request { }
I20260810 07:53:03.896068 1880 raft_consensus.cc:2889] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:53:03.896431 1880 raft_consensus.cc:2272] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:53:03.894670 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 62720.1us
I20260810 07:53:03.915925 1071 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:03.916497 1071 ts_tablet_manager.cc:1403] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.125s user 0.008s sys 0.009s
I20260810 07:53:03.923720 1969 leader_election.cc:290] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.933385 1880 ts_tablet_manager.cc:1916] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:03.844940 32460 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_956f0c9ce19f4e929b17f81669a8e797" } new_table_name: "default.Table_d8243e06c4324d7ba096253015b2eebd" modify_external_catalogs: true
W20260810 07:53:03.938215 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 101199us
W20260810 07:53:03.938987 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because the tablet was not found. No further retry: Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
W20260810 07:53:03.939674 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 09712cfebab34e56ba844e7aa0e8d152 because the tablet was not found. No further retry: Not found: Tablet not found: 09712cfebab34e56ba844e7aa0e8d152
I20260810 07:53:03.939931 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd22d1029ff249a0a9e05862293261ff" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:03.940291 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 4ec00a494816409684205c7f4435a079 because the tablet was not found. No further retry: Not found: Tablet not found: 4ec00a494816409684205c7f4435a079
I20260810 07:53:03.940621 415 raft_consensus.cc:2468] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:03.942665 1671 raft_consensus.cc:493] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:03.943210 1671 raft_consensus.cc:515] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.947234 1671 leader_election.cc:290] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:03.947516 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 08520a2ac0f64c3abd534a32100839a4 because the tablet was not found. No further retry: Not found: Tablet not found: 08520a2ac0f64c3abd534a32100839a4
W20260810 07:53:03.948220 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 92e2678ab165486dbbb69c7c183a7fa4 because the tablet was not found. No further retry: Not found: Tablet not found: 92e2678ab165486dbbb69c7c183a7fa4
I20260810 07:53:03.949469 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet c1faed468bfa4d1ba5d92c0b9b194781 (table default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]) successfully deleted
I20260810 07:53:03.952481 369 tablet_service.cc:1511] Processing CreateTablet for tablet f4a46a037cae4f599398b289bb6a2d84 (DEFAULT_TABLE table=default.Table_d4c51e03eea74f81a39e48e631649307 [id=466e3662813242f9b2c85763c28ae42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.953922 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a46a037cae4f599398b289bb6a2d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.958583 645 tablet_service.cc:1511] Processing CreateTablet for tablet f4a46a037cae4f599398b289bb6a2d84 (DEFAULT_TABLE table=default.Table_d4c51e03eea74f81a39e48e631649307 [id=466e3662813242f9b2c85763c28ae42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.959802 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a46a037cae4f599398b289bb6a2d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.974170 380 tablet_service.cc:1511] Processing CreateTablet for tablet a4863f3f03104d31ad4dafda6f33c9af (DEFAULT_TABLE table=default.Table_712d9cfefe684238a26898997e816b64 [id=8753c36aa07540b68965b2a61a467bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.974464 623 tablet_service.cc:1511] Processing CreateTablet for tablet a4863f3f03104d31ad4dafda6f33c9af (DEFAULT_TABLE table=default.Table_712d9cfefe684238a26898997e816b64 [id=8753c36aa07540b68965b2a61a467bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.975467 620 tablet_service.cc:1558] Processing DeleteTablet for tablet a89cdc44a8c64ccd83fd30855798f87b with delete_type TABLET_DATA_DELETED (replaced by tablet aed14fa26b9a46a4b3e9a43bac05a795) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:03.976325 376 tablet_service.cc:1558] Processing DeleteTablet for tablet a89cdc44a8c64ccd83fd30855798f87b with delete_type TABLET_DATA_DELETED (replaced by tablet aed14fa26b9a46a4b3e9a43bac05a795) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:03.976696 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4863f3f03104d31ad4dafda6f33c9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.980701 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 8d717f39a6704199abc34b220d795e48 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:03.985131 376 tablet_service.cc:1511] Processing CreateTablet for tablet a4d1c8d270f744ee9849475c9128414b (DEFAULT_TABLE table=default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:03.986161 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d1c8d270f744ee9849475c9128414b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:03.929936 1071 raft_consensus.cc:359] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.991695 1071 raft_consensus.cc:385] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:03.992841 1071 raft_consensus.cc:740] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:03.993598 1071 consensus_queue.cc:260] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.000592 635 tablet_service.cc:1511] Processing CreateTablet for tablet a4d1c8d270f744ee9849475c9128414b (DEFAULT_TABLE table=default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.003469 375 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb896c8922e412c97e2cc4891298830 (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.004741 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb896c8922e412c97e2cc4891298830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.012681 1071 ts_tablet_manager.cc:1434] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.096s user 0.003s sys 0.004s
I20260810 07:53:04.023344 1071 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.022679 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:04.030588 620 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb896c8922e412c97e2cc4891298830 (DEFAULT_TABLE table=default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.033454 371 tablet_service.cc:1511] Processing CreateTablet for tablet e1fa9200140a41dfbd886442ae336988 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.034582 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fa9200140a41dfbd886442ae336988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.034475 32465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c56854332dfd4628a280b1d87bf7a2cb" } new_table_name: "default.Table_77e86fb7c5274253ab027b7dec950ae0" modify_external_catalogs: true
I20260810 07:53:04.042189 1880 ts_tablet_manager.cc:1929] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:04.042579 1880 log.cc:1199] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:53:04.043314 1880 ts_tablet_manager.cc:1950] T 2c171d3358e44ae2962ab3c4c377b0f1 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:04.046625 622 tablet_service.cc:1511] Processing CreateTablet for tablet e1fa9200140a41dfbd886442ae336988 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.048413 1071 tablet_bootstrap.cc:654] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.052686 380 tablet_service.cc:1511] Processing CreateTablet for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d (DEFAULT_TABLE table=default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.054035 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.059996 1880 tablet_replica.cc:333] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:04.060937 1880 raft_consensus.cc:2243] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:04.061422 1880 raft_consensus.cc:2272] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:04.062070 633 tablet_service.cc:1511] Processing CreateTablet for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d (DEFAULT_TABLE table=default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.062781 382 tablet_service.cc:1511] Processing CreateTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 (DEFAULT_TABLE table=default.Table_eb0339f23b7749ccb8c8081ee9dcae9d [id=caa8fc2fafc14076aaf44b9056c04470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.063606 627 tablet_service.cc:1511] Processing CreateTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 (DEFAULT_TABLE table=default.Table_eb0339f23b7749ccb8c8081ee9dcae9d [id=caa8fc2fafc14076aaf44b9056c04470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.063967 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fa93c49cb54e5fa03034eda6fc2300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.065143 573 leader_election.cc:304] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:04.066222 1600 raft_consensus.cc:2804] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:04.066601 1600 raft_consensus.cc:493] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:04.066926 1600 raft_consensus.cc:3060] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.070164 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4863f3f03104d31ad4dafda6f33c9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.072679 2106 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 218) took 1113 ms. Trace:
I20260810 07:53:04.073069 2106 rpcz_store.cc:276] 0810 07:53:02.959178 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.043556 (+ 84378us) service_pool.cc:224] Handling call
0810 07:53:04.072600 (+1029044us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:03.044701 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:03.045113 (+ 412us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:03.045148 (+ 35us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:03.045266 (+ 118us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:03.045391 (+ 125us) alter_schema_op.cc:128] START. Timestamp: 1786348383044784
0810 07:53:03.045420 (+ 29us) op_driver.cc:348] REPLICATION: starting
0810 07:53:03.045958 (+ 538us) log.cc:844] Serialized 173 byte log entry
0810 07:53:03.227271 (+181313us) op_driver.cc:464] REPLICATION: finished
0810 07:53:03.541236 (+313965us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:03.543318 (+ 2082us) log.cc:844] Serialized 5 byte log entry
0810 07:53:03.927238 (+383920us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:03.928338 (+ 1100us) log.cc:844] Serialized 14 byte log entry
0810 07:53:04.072423 (+144085us) alter_schema_op.cc:73] Released schema lock
0810 07:53:04.072475 (+ 52us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":313714,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":1630,"prepare.run_wall_time_us":7501,"raft.queue_time_us":1818,"raft.run_cpu_time_us":1626,"raft.run_wall_time_us":2006,"replication_time_us":181707,"thread_start_us":242856,"threads_started":1}]]}
I20260810 07:53:04.081853 1600 raft_consensus.cc:515] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.090059 1880 ts_tablet_manager.cc:1916] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:04.094873 1047 tablet_bootstrap.cc:654] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.098851 1071 tablet_bootstrap.cc:492] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.099311 1071 ts_tablet_manager.cc:1403] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.000s
I20260810 07:53:04.101605 1071 raft_consensus.cc:359] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.102238 1071 raft_consensus.cc:385] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.102502 1071 raft_consensus.cc:740] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:04.088480 573 leader_election.cc:343] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:04.113119 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d1c8d270f744ee9849475c9128414b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.118417 1071 consensus_queue.cc:260] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.119735 376 tablet_service.cc:1511] Processing CreateTablet for tablet 6db16797cabc48fe8ec82355ce3f16be (DEFAULT_TABLE table=default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.121014 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db16797cabc48fe8ec82355ce3f16be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.121107 385 tablet_service.cc:1511] Processing CreateTablet for tablet fe8b4892a53742f48dd43397c4f8e88f (DEFAULT_TABLE table=default.Table_3cec5176ba3946aebc2391756561ecc1 [id=1bba437bed3d48c39bc2c08e6c3a5072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.122656 367 tablet_service.cc:1511] Processing CreateTablet for tablet 0d3aaf0099934d1e9bede4687b36ce44 (DEFAULT_TABLE table=default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.122958 375 tablet_service.cc:1511] Processing CreateTablet for tablet 849197942f134542a387a10167fc7481 (DEFAULT_TABLE table=default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.123811 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8b4892a53742f48dd43397c4f8e88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.123889 371 tablet_service.cc:1558] Processing DeleteTablet for tablet fe8b4892a53742f48dd43397c4f8e88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:04.125232 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849197942f134542a387a10167fc7481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.129022 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3aaf0099934d1e9bede4687b36ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.150817 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 5e4b3ca46c9c44559761e25bcde3e927 (table default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]) successfully deleted
I20260810 07:53:04.151819 382 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea09585009e4a3ea33f620182bfee60 (DEFAULT_TABLE table=default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.152967 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea09585009e4a3ea33f620182bfee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.165954 1071 ts_tablet_manager.cc:1434] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.066s user 0.007s sys 0.000s
I20260810 07:53:04.170670 1071 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.194489 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 2c171d3358e44ae2962ab3c4c377b0f1 (table default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]) successfully deleted
W20260810 07:53:04.195452 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet d7626d30f4574658bb226f2e915ee0de because the tablet was not found. No further retry: Not found: Tablet not found: d7626d30f4574658bb226f2e915ee0de
I20260810 07:53:04.206650 792 leader_election.cc:304] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:04.209383 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 429616us
I20260810 07:53:04.217108 32467 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a4863f3f03104d31ad4dafda6f33c9af
I20260810 07:53:04.240588 407 raft_consensus.cc:3060] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.248063 620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb896c8922e412c97e2cc4891298830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.248436 624 tablet_service.cc:1511] Processing CreateTablet for tablet 6db16797cabc48fe8ec82355ce3f16be (DEFAULT_TABLE table=default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.249393 32467 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_90bfccd382de4eebb7f693185dab51d3" } modify_external_catalogs: true
I20260810 07:53:04.249687 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db16797cabc48fe8ec82355ce3f16be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.249944 32467 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_90bfccd382de4eebb7f693185dab51d3" } modify_external_catalogs: true
I20260810 07:53:04.252952 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_20e00f06bba84d53bf31edd76a1a3940"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:04.254936 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20e00f06bba84d53bf31edd76a1a3940 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:04.257170 1600 leader_election.cc:290] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:04.257337 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1fa9200140a41dfbd886442ae336988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.261312 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.264418 1047 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:04.264953 1047 ts_tablet_manager.cc:1403] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.441s user 0.012s sys 0.000s
I20260810 07:53:04.265291 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fa93c49cb54e5fa03034eda6fc2300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.267638 1047 raft_consensus.cc:359] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.268255 1047 raft_consensus.cc:385] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.269415 376 tablet_service.cc:1558] Processing DeleteTablet for tablet a4863f3f03104d31ad4dafda6f33c9af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:04.277855 410 raft_consensus.cc:3060] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.268522 1047 raft_consensus.cc:740] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.289330 1047 consensus_queue.cc:260] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.299391 1969 consensus_queue.cc:1048] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:04.313644 1047 ts_tablet_manager.cc:1434] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260810 07:53:04.325409 645 tablet_service.cc:1511] Processing CreateTablet for tablet 849197942f134542a387a10167fc7481 (DEFAULT_TABLE table=default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.326555 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849197942f134542a387a10167fc7481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.327054 619 tablet_service.cc:1511] Processing CreateTablet for tablet 0d3aaf0099934d1e9bede4687b36ce44 (DEFAULT_TABLE table=default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.327466 631 tablet_service.cc:1511] Processing CreateTablet for tablet fe8b4892a53742f48dd43397c4f8e88f (DEFAULT_TABLE table=default.Table_3cec5176ba3946aebc2391756561ecc1 [id=1bba437bed3d48c39bc2c08e6c3a5072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.328171 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3aaf0099934d1e9bede4687b36ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.328541 646 tablet_service.cc:1558] Processing DeleteTablet for tablet a4863f3f03104d31ad4dafda6f33c9af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:04.329391 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8b4892a53742f48dd43397c4f8e88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.326421 630 tablet_service.cc:1558] Processing DeleteTablet for tablet fe8b4892a53742f48dd43397c4f8e88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:04.339789 849 consensus_meta.cc:287] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.653s user 0.014s sys 0.000s
I20260810 07:53:04.341774 851 tablet_service.cc:1558] Processing DeleteTablet for tablet a4863f3f03104d31ad4dafda6f33c9af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:04.344786 852 consensus_meta.cc:287] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.538s user 0.009s sys 0.002s
W20260810 07:53:04.351058 855 consensus_meta.cc:287] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.615s user 0.004s sys 0.008s
I20260810 07:53:04.292687 1071 tablet_bootstrap.cc:654] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:03.832334 856 tablet_service.cc:1511] Processing CreateTablet for tablet fe8b4892a53742f48dd43397c4f8e88f (DEFAULT_TABLE table=default.Table_3cec5176ba3946aebc2391756561ecc1 [id=1bba437bed3d48c39bc2c08e6c3a5072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.355991 859 tablet_service.cc:1511] Processing CreateTablet for tablet 0d3aaf0099934d1e9bede4687b36ce44 (DEFAULT_TABLE table=default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.336263 2171 raft_consensus.cc:2749] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:04.299201 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd22d1029ff249a0a9e05862293261ff" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:04.357558 410 raft_consensus.cc:3060] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:03.872526 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db16797cabc48fe8ec82355ce3f16be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.360040 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8b4892a53742f48dd43397c4f8e88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.365144 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d3aaf0099934d1e9bede4687b36ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.316473 2169 tablet.cc:1724] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:03.788203 864 tablet_service.cc:1558] Processing DeleteTablet for tablet fe8b4892a53742f48dd43397c4f8e88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:03 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:03.780700 866 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea09585009e4a3ea33f620182bfee60 (DEFAULT_TABLE table=default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.382361 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849197942f134542a387a10167fc7481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.382221 410 raft_consensus.cc:2468] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:04.383399 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea09585009e4a3ea33f620182bfee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.398116 883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:04.399295 883 raft_consensus.cc:2468] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:04.412773 2179 tablet.cc:1724] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:04.422638 1880 ts_tablet_manager.cc:1929] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:04.423182 1880 log.cc:1199] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:04.424036 1880 ts_tablet_manager.cc:1950] T a89cdc44a8c64ccd83fd30855798f87b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:04.417457 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:04.423796 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a3bd8369634e4bbfdf8920454357bc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:04.428347 2176 raft_consensus.cc:493] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:04.428601 892 raft_consensus.cc:3060] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.429250 2176 raft_consensus.cc:515] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:03.787065 1064 tablet_bootstrap.cc:654] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.440665 892 raft_consensus.cc:2468] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:04.451109 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_6261a75ba7d54f049285cbd6d8d5b878"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:04.453009 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6261a75ba7d54f049285cbd6d8d5b878 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:04.458413 623 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea09585009e4a3ea33f620182bfee60 (DEFAULT_TABLE table=default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:04.459658 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea09585009e4a3ea33f620182bfee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:04.461768 1047 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:04.462695 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 110184us
I20260810 07:53:04.477972 2106 tablet.cc:1724] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:04.506482 1671 raft_consensus.cc:493] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:04.507000 1671 raft_consensus.cc:515] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.509752 1671 leader_election.cc:290] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:03.868217 2088 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:04.520253 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 consensus state has no leader
I20260810 07:53:04.529156 1047 tablet_bootstrap.cc:654] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:04.531611 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 68013.1us
I20260810 07:53:04.532361 573 leader_election.cc:304] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:04.542361 2088 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:04.543426 896 raft_consensus.cc:2393] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
W20260810 07:53:04.556075 32465 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e2fe3153512b40a99caf3fb9b81fd7a1 consensus state has no leader
I20260810 07:53:04.432652 2176 leader_election.cc:290] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:04.561316 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 8d717f39a6704199abc34b220d795e48
I20260810 07:53:04.561957 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89cdc44a8c64ccd83fd30855798f87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:04.570609 32464 catalog_manager.cc:6807] ReplaceTablet: tablet 8d717f39a6704199abc34b220d795e48 deleted and replaced by tablet 94ced893d6804aa395b73f1f5ed085f9
I20260810 07:53:04.518179 1071 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.572736 1071 ts_tablet_manager.cc:1403] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.402s user 0.014s sys 0.000s
I20260810 07:53:04.575466 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet a89cdc44a8c64ccd83fd30855798f87b (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
I20260810 07:53:04.575307 1071 raft_consensus.cc:359] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.575945 1071 raft_consensus.cc:385] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:53:04.576153 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet fe8b4892a53742f48dd43397c4f8e88f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8b4892a53742f48dd43397c4f8e88f already in progress: creating tablet
I20260810 07:53:04.576191 1071 raft_consensus.cc:740] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.577302 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d717f39a6704199abc34b220d795e48 with delete_type TABLET_DATA_DELETED (replaced by tablet 94ced893d6804aa395b73f1f5ed085f9) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:04.578650 631 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d717f39a6704199abc34b220d795e48 with delete_type TABLET_DATA_DELETED (replaced by tablet 94ced893d6804aa395b73f1f5ed085f9) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:04.579873 573 leader_election.cc:343] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260810 07:53:04.590831 2120 consensus_queue.cc:1048] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:04.588445 1671 raft_consensus.cc:2804] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:04.591655 1671 raft_consensus.cc:697] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:04.592468 1671 consensus_queue.cc:237] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.595778 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_868b99bda3b04eafab388ae9cbf490fd" } modify_external_catalogs: true
I20260810 07:53:04.595866 1071 consensus_queue.cc:260] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.596655 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_868b99bda3b04eafab388ae9cbf490fd" } modify_external_catalogs: true
I20260810 07:53:04.598971 1071 ts_tablet_manager.cc:1434] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.026s user 0.005s sys 0.002s
I20260810 07:53:04.599767 1071 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.603075 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_15c4a71f6b934bd58563002b8e876760"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:04.605722 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15c4a71f6b934bd58563002b8e876760 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:04.615298 1071 tablet_bootstrap.cc:654] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:04.607627 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet a4863f3f03104d31ad4dafda6f33c9af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4863f3f03104d31ad4dafda6f33c9af already in progress: creating tablet
I20260810 07:53:04.632278 1071 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.633841 1071 ts_tablet_manager.cc:1403] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.000s
W20260810 07:53:04.634857 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 8d717f39a6704199abc34b220d795e48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d717f39a6704199abc34b220d795e48 already in progress: creating tablet
I20260810 07:53:04.636950 849 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d717f39a6704199abc34b220d795e48 with delete_type TABLET_DATA_DELETED (replaced by tablet 94ced893d6804aa395b73f1f5ed085f9) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:04.640111 1071 raft_consensus.cc:359] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.640841 1071 raft_consensus.cc:385] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.641186 1071 raft_consensus.cc:740] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.642242 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe1a6581374213be7c3f63a7a567a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:04.643018 896 raft_consensus.cc:2393] T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:04.645067 1071 consensus_queue.cc:260] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.647043 1071 ts_tablet_manager.cc:1434] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260810 07:53:04.648079 1071 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.654470 2092 ts_tablet_manager.cc:1929] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:04.685401 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:04.689234 309 leader_election.cc:304] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:04.690966 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:04.694412 2103 raft_consensus.cc:2749] T 0dbe1a6581374213be7c3f63a7a567a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:04.697659 792 leader_election.cc:343] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:04.699296 792 consensus_peers.cc:597] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:04.699955 1874 ts_tablet_manager.cc:1929] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:04.700423 1874 log.cc:1199] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/5e4b3ca46c9c44559761e25bcde3e927
I20260810 07:53:04.701413 1874 ts_tablet_manager.cc:1950] T 5e4b3ca46c9c44559761e25bcde3e927 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:04.702322 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 194) took 2446 ms. Trace:
I20260810 07:53:04.702692 1874 rpcz_store.cc:276] 0810 07:53:02.256236 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.265187 (+ 8951us) service_pool.cc:224] Handling call
0810 07:53:03.724337 (+1459150us) ts_tablet_manager.cc:1151] Deleting tablet 5e4b3ca46c9c44559761e25bcde3e927
0810 07:53:03.724384 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:04.583115 (+858731us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.635908 (+ 52793us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.702235 (+ 66327us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1440156}
I20260810 07:53:04.704409 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 5e4b3ca46c9c44559761e25bcde3e927 (table default.Table_3523dbadfd954f2da7cd8e456fffa4c4 [id=74f74adc5c744b70bf64b01e78bc2b4c]) successfully deleted
I20260810 07:53:04.706686 1071 tablet_bootstrap.cc:654] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.699546 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:04.708287 676 raft_consensus.cc:2393] T ab2e7619a25b437c8ee2c817255ca638 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:04.700824 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:04.706185 1047 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:04.711642 1047 ts_tablet_manager.cc:1403] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.250s user 0.023s sys 0.021s
I20260810 07:53:04.718928 1047 raft_consensus.cc:359] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.719789 1047 raft_consensus.cc:385] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.739996 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89cdc44a8c64ccd83fd30855798f87b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:04.746150 1064 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:04.746803 1064 ts_tablet_manager.cc:1403] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 1.048s user 0.012s sys 0.012s
I20260810 07:53:04.766520 1071 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.766984 1071 ts_tablet_manager.cc:1403] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.119s user 0.012s sys 0.000s
I20260810 07:53:04.769073 1047 raft_consensus.cc:740] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.720181 2092 log.cc:1199] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:04.770995 2092 ts_tablet_manager.cc:1950] T 37b48c88571040f6be8d7cbd388feb58 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:04.776605 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 129711us
I20260810 07:53:04.777421 574 leader_election.cc:304] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:04.769801 1047 consensus_queue.cc:260] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:04.790362 574 leader_election.cc:343] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:04.776508 1071 raft_consensus.cc:359] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.793601 1071 raft_consensus.cc:385] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.794041 1071 raft_consensus.cc:740] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.794866 1071 consensus_queue.cc:260] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.798838 1874 tablet_replica.cc:333] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:04.801013 1874 raft_consensus.cc:2243] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:04.805858 866 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55008 (request call id 222) took 1026 ms. Trace:
I20260810 07:53:04.806211 866 rpcz_store.cc:276] 0810 07:53:03.779971 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.780109 (+ 138us) service_pool.cc:224] Handling call
0810 07:53:04.382686 (+602577us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:04.382825 (+ 139us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:04.675635 (+292810us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.805314 (+129679us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:04.805795 (+ 481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:04.808709 1071 ts_tablet_manager.cc:1434] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.041s user 0.010s sys 0.000s
I20260810 07:53:04.818436 1071 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.822943 792 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:04.818596 2120 raft_consensus.cc:2804] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:04.835551 2120 raft_consensus.cc:493] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:04.835924 2120 raft_consensus.cc:3060] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.838476 1874 raft_consensus.cc:2272] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:04.845798 2173 raft_consensus.cc:2749] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:04.847759 1064 raft_consensus.cc:359] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.848523 1064 raft_consensus.cc:385] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.848847 1064 raft_consensus.cc:740] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.849491 1064 consensus_queue.cc:260] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.844527 1874 ts_tablet_manager.cc:1916] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:04.859258 792 leader_election.cc:343] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:04.859689 792 leader_election.cc:304] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:04.860844 2173 raft_consensus.cc:2749] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:04.866060 1064 ts_tablet_manager.cc:1434] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.119s user 0.003s sys 0.004s
W20260810 07:53:04.871029 574 leader_election.cc:343] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:04.873191 1071 tablet_bootstrap.cc:654] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.873257 1047 ts_tablet_manager.cc:1434] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.160s user 0.007s sys 0.000s
W20260810 07:53:04.882344 574 leader_election.cc:343] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:04.886727 2103 raft_consensus.cc:493] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:04.887286 2103 raft_consensus.cc:515] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.889618 2103 leader_election.cc:290] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:04.892601 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:04.893539 893 raft_consensus.cc:2468] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:04.895556 309 leader_election.cc:304] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:04.896974 1064 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:04.897825 2103 raft_consensus.cc:2804] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:04.898301 2103 raft_consensus.cc:493] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:04.898962 2103 raft_consensus.cc:3060] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:04.906750 1071 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.907248 1071 ts_tablet_manager.cc:1403] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.089s user 0.011s sys 0.003s
I20260810 07:53:04.910907 1071 raft_consensus.cc:359] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.911662 1071 raft_consensus.cc:385] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.911944 1071 raft_consensus.cc:740] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.912810 1071 consensus_queue.cc:260] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.924664 1064 tablet_bootstrap.cc:654] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.929428 1071 ts_tablet_manager.cc:1434] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260810 07:53:04.930644 1071 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:04.907811 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:04.936583 308 leader_election.cc:343] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:04.936738 574 leader_election.cc:304] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:04.940204 1969 raft_consensus.cc:2749] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:04.940331 2225 consensus_queue.cc:1048] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:04.883596 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 220) took 1854 ms. Trace:
I20260810 07:53:04.941951 2092 rpcz_store.cc:276] 0810 07:53:03.029114 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.029231 (+ 117us) service_pool.cc:224] Handling call
0810 07:53:03.833489 (+804258us) ts_tablet_manager.cc:1151] Deleting tablet 37b48c88571040f6be8d7cbd388feb58
0810 07:53:03.833538 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:04.298042 (+464504us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.491318 (+193276us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.881934 (+390616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3959,"tablet-delete.queue_time_us":803720}
I20260810 07:53:04.951171 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 37b48c88571040f6be8d7cbd388feb58 (table default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]) successfully deleted
I20260810 07:53:04.953938 1071 tablet_bootstrap.cc:654] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:04.922114 1047 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:04.955475 1874 ts_tablet_manager.cc:1929] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:04.955909 1874 log.cc:1199] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:04.956840 1874 ts_tablet_manager.cc:1950] T 37b48c88571040f6be8d7cbd388feb58 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:04.959242 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 195) took 2593 ms. Trace:
I20260810 07:53:04.973894 1874 rpcz_store.cc:276] 0810 07:53:02.365570 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.365788 (+ 218us) service_pool.cc:224] Handling call
0810 07:53:04.798595 (+2432807us) ts_tablet_manager.cc:1151] Deleting tablet 37b48c88571040f6be8d7cbd388feb58
0810 07:53:04.798692 (+ 97us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:04.891645 (+ 92953us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.936597 (+ 44952us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:04.959177 (+ 22580us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36319,"tablet-delete.queue_time_us":2319830}
I20260810 07:53:04.975843 1064 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:04.975795 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 37b48c88571040f6be8d7cbd388feb58 (table default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]) successfully deleted
I20260810 07:53:04.976320 1064 ts_tablet_manager.cc:1403] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.006s
I20260810 07:53:04.976935 1874 tablet_replica.cc:333] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:04.977885 1874 raft_consensus.cc:2243] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:04.978533 1874 raft_consensus.cc:2272] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:04.979146 1064 raft_consensus.cc:359] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.979913 1064 raft_consensus.cc:385] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.980232 1064 raft_consensus.cc:740] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.982115 1071 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:04.982520 1071 ts_tablet_manager.cc:1403] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.052s user 0.011s sys 0.000s
I20260810 07:53:04.985755 1071 raft_consensus.cc:359] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.986678 1071 raft_consensus.cc:385] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:04.987071 1071 raft_consensus.cc:740] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:04.989351 1064 consensus_queue.cc:260] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.991927 1064 ts_tablet_manager.cc:1434] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.015s user 0.003s sys 0.004s
I20260810 07:53:04.995834 1874 ts_tablet_manager.cc:1916] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:04.953990 2092 tablet_replica.cc:333] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:05.000623 2092 raft_consensus.cc:2243] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:05.001173 1064 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.001840 2092 raft_consensus.cc:2272] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.022629 1064 tablet_bootstrap.cc:654] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.032711 1071 consensus_queue.cc:260] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:04.983461 2120 raft_consensus.cc:515] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.035624 2120 leader_election.cc:290] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:04.937767 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 55449.7us
I20260810 07:53:05.051225 1969 consensus_queue.cc:1048] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:05.055089 1064 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.055579 1064 ts_tablet_manager.cc:1403] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.055s user 0.013s sys 0.000s
I20260810 07:53:05.058136 1064 raft_consensus.cc:359] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.058768 1064 raft_consensus.cc:385] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.066924 1064 raft_consensus.cc:740] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.070195 2230 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:05.071754 2230 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:05.077010 1064 consensus_queue.cc:260] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.079732 1064 ts_tablet_manager.cc:1434] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.024s user 0.008s sys 0.002s
I20260810 07:53:05.080806 1064 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:05.086845 32565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
I20260810 07:53:05.088949 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:05.090624 32462 catalog_manager.cc:6807] ReplaceTablet: tablet acc89d15b4d544b7bc8abc12a4c6d3e5 deleted and replaced by tablet 04aff252a46447f49c59345035374580
I20260810 07:53:05.091023 32462 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 266) took 1391 ms. Trace:
I20260810 07:53:05.091586 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:05.092514 32470 catalog_manager.cc:5719] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:05.094074 866 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:05.094732 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:05.099073 32470 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:05.091333 32462 rpcz_store.cc:276] 0810 07:53:03.699492 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.699734 (+ 242us) service_pool.cc:224] Handling call
0810 07:53:05.090953 (+1391219us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.650255 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.650644 (+ 389us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.650668 (+ 24us) write_op.cc:435] Acquired schema lock
0810 07:53:04.650681 (+ 13us) tablet.cc:662] Decoding operations
0810 07:53:04.650981 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.651019 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.651042 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:04.651228 (+ 186us) tablet.cc:701] Row locks acquired
0810 07:53:04.651241 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:04.651343 (+ 102us) write_op.cc:270] Start()
0810 07:53:04.651430 (+ 87us) write_op.cc:276] Timestamp: P: 1786348384651293 usec, L: 0
0810 07:53:04.651446 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.655056 (+ 3610us) log.cc:844] Serialized 662 byte log entry
0810 07:53:05.081389 (+426333us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.081696 (+ 307us) write_op.cc:301] APPLY: starting
0810 07:53:05.081754 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.081861 (+ 107us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.081980 (+ 119us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.082367 (+ 387us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.082605 (+ 238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:05.082649 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:05.082871 (+ 222us) log.cc:844] Serialized 29 byte log entry
0810 07:53:05.083035 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.083104 (+ 69us) write_op.cc:454] Released schema lock
0810 07:53:05.083135 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":4328,"child_traces":[["op",{"apply.queue_time_us":143,"apply.run_cpu_time_us":1674,"apply.run_wall_time_us":1674,"num_ops":2,"prepare.queue_time_us":418624,"prepare.run_cpu_time_us":2227,"prepare.run_wall_time_us":85390,"raft.queue_time_us":78052,"raft.run_cpu_time_us":847,"raft.run_wall_time_us":1267,"replication_time_us":429833}]]}
I20260810 07:53:05.104494 1064 tablet_bootstrap.cc:654] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.106328 2230 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:05.108218 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 318 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:05.125471 1874 ts_tablet_manager.cc:1929] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:05.131793 1874 log.cc:1199] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/d7626d30f4574658bb226f2e915ee0de
I20260810 07:53:05.135151 1874 ts_tablet_manager.cc:1950] T d7626d30f4574658bb226f2e915ee0de P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:05.143059 2147 raft_consensus.cc:493] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.145960 2147 raft_consensus.cc:515] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.148214 2147 leader_election.cc:290] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.151245 1064 tablet_bootstrap.cc:492] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.151834 1064 ts_tablet_manager.cc:1403] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.001s
I20260810 07:53:05.059039 1047 tablet_bootstrap.cc:654] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.154964 32467 catalog_manager.cc:6001] Not sending DeleteTablet requests; no consensus state for tablet 554ff75a99294cbab7362ec1185fd2c4
I20260810 07:53:05.111634 623 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:05.155340 32467 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 271) took 1165 ms. Trace:
W20260810 07:53:05.157696 573 consensus_peers.cc:597] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1faed468bfa4d1ba5d92c0b9b194781. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:05.154553 1064 raft_consensus.cc:359] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.158106 1064 raft_consensus.cc:385] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.158419 1064 raft_consensus.cc:740] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.155601 32467 rpcz_store.cc:276] 0810 07:53:03.989520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.218896 (+229376us) service_pool.cc:224] Handling call
0810 07:53:04.249280 (+ 30384us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c4d6 00007f30cf424b34 00007f30cf424ac2 00007f30cf4249f1 00007f30cf42496c 00007f30cf423c22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:04.250338 (+ 1058us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:04.250826 (+ 488us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:04.250950 (+ 124us) catalog_manager.cc:2850] Locking tablets
0810 07:53:04.251432 (+ 482us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:05.154619 (+903187us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:05.154789 (+ 170us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:05.154874 (+ 85us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:05.155286 (+ 412us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.735788 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.736258 (+ 470us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.736288 (+ 30us) write_op.cc:435] Acquired schema lock
0810 07:53:04.736305 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:04.737098 (+ 793us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.737136 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.737161 (+ 25us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:04.737380 (+ 219us) tablet.cc:701] Row locks acquired
0810 07:53:04.737394 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:04.737514 (+ 120us) write_op.cc:270] Start()
0810 07:53:04.737606 (+ 92us) write_op.cc:276] Timestamp: P: 1786348384737451 usec, L: 0
0810 07:53:04.737624 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.738234 (+ 610us) log.cc:844] Serialized 533 byte log entry
0810 07:53:05.151550 (+413316us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.153186 (+ 1636us) write_op.cc:301] APPLY: starting
0810 07:53:05.153257 (+ 71us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.153375 (+ 118us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.153525 (+ 150us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.153801 (+ 276us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.153954 (+ 153us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:05.153998 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:05.154181 (+ 183us) log.cc:844] Serialized 29 byte log entry
0810 07:53:05.154338 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.154398 (+ 60us) write_op.cc:454] Released schema lock
0810 07:53:05.154431 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":67259136,"child_traces":[["op",{"apply.queue_time_us":1456,"num_ops":2,"prepare.queue_time_us":472765,"prepare.run_cpu_time_us":2486,"prepare.run_wall_time_us":46199,"raft.queue_time_us":150,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":17,"replication_time_us":413801}]]}
I20260810 07:53:05.161348 1064 consensus_queue.cc:260] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.164103 1064 ts_tablet_manager.cc:1434] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20260810 07:53:05.165377 1064 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.180394 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:05.180828 2229 raft_consensus.cc:493] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.180986 896 raft_consensus.cc:3060] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.181305 2229 raft_consensus.cc:515] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.184130 2229 leader_election.cc:290] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.184612 2229 raft_consensus.cc:493] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.185006 32468 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 272) took 1008 ms. Trace:
I20260810 07:53:05.185052 2229 raft_consensus.cc:515] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.190326 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.191336 1064 tablet_bootstrap.cc:654] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:05.191833 792 leader_election.cc:343] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.194113 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.194869 877 raft_consensus.cc:2468] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:05.196655 309 leader_election.cc:304] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:05.203548 1064 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.204020 1064 ts_tablet_manager.cc:1403] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.004s
I20260810 07:53:05.185431 32468 rpcz_store.cc:276] 0810 07:53:04.176878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.228233 (+ 51355us) service_pool.cc:224] Handling call
0810 07:53:04.248146 (+ 19913us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c1c6 00007f30cf420034 00007f30cf41ffc2 00007f30cf41fef1 00007f30cf41fe6c 00007f30cf41f122 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:04.255649 (+ 7503us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:04.273775 (+ 18126us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:05.184263 (+910488us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:05.184549 (+ 286us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:05.184911 (+ 362us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.782125 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.782611 (+ 486us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.782639 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:04.782656 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:04.783118 (+ 462us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.783162 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.783189 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:04.783500 (+ 311us) tablet.cc:701] Row locks acquired
0810 07:53:04.783517 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:53:04.783640 (+ 123us) write_op.cc:270] Start()
0810 07:53:04.783736 (+ 96us) write_op.cc:276] Timestamp: P: 1786348384783568 usec, L: 0
0810 07:53:04.783755 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.784367 (+ 612us) log.cc:844] Serialized 433 byte log entry
0810 07:53:05.151808 (+367441us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.159489 (+ 7681us) write_op.cc:301] APPLY: starting
0810 07:53:05.159541 (+ 52us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.159642 (+ 101us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.159731 (+ 89us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.160131 (+ 400us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.160268 (+ 137us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:05.160323 (+ 55us) write_op.cc:312] APPLY: finished
0810 07:53:05.160495 (+ 172us) log.cc:844] Serialized 29 byte log entry
0810 07:53:05.160653 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.160735 (+ 82us) write_op.cc:454] Released schema lock
0810 07:53:05.160765 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":41913216,"child_traces":[["op",{"apply.queue_time_us":7536,"apply.run_cpu_time_us":1507,"apply.run_wall_time_us":23349,"num_ops":2,"prepare.queue_time_us":507177,"prepare.run_cpu_time_us":2515,"prepare.run_wall_time_us":30025,"raft.queue_time_us":119,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":367970,"threads_started":1}]]}
I20260810 07:53:05.209175 1064 raft_consensus.cc:359] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.210062 1064 raft_consensus.cc:385] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.210467 1064 raft_consensus.cc:740] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.211215 1064 consensus_queue.cc:260] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.213521 1064 ts_tablet_manager.cc:1434] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20260810 07:53:05.214421 1064 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.214200 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.214933 669 raft_consensus.cc:2393] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:05.219904 2229 leader_election.cc:290] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.222900 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.223127 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.229132 2229 raft_consensus.cc:2804] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.229662 2229 raft_consensus.cc:493] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.230031 2229 raft_consensus.cc:3060] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.230320 2263 tablet.cc:1724] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260810 07:53:05.233147 308 leader_election.cc:343] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.233208 309 leader_election.cc:304] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.243279 896 raft_consensus.cc:2468] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:05.229902 1064 tablet_bootstrap.cc:654] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.149353 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:05.248968 792 leader_election.cc:343] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.249346 792 leader_election.cc:304] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.250387 2147 raft_consensus.cc:2749] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.004623 2092 ts_tablet_manager.cc:1916] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.280282 2147 raft_consensus.cc:493] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.267608 1047 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:05.281107 1047 ts_tablet_manager.cc:1403] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.398s user 0.020s sys 0.000s
I20260810 07:53:05.280979 2147 raft_consensus.cc:515] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.283170 2147 leader_election.cc:290] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.283104 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:05.283674 415 raft_consensus.cc:3060] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.285308 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.286408 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:05.288136 792 leader_election.cc:343] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.290683 2220 consensus_queue.cc:1048] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:05.292853 1071 ts_tablet_manager.cc:1434] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.310s user 0.012s sys 0.010s
W20260810 07:53:05.296478 792 leader_election.cc:343] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.148551 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 196) took 2741 ms. Trace:
I20260810 07:53:05.297453 792 leader_election.cc:304] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.301712 2147 raft_consensus.cc:2749] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:05.302248 32565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 215584us
I20260810 07:53:05.309337 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_6cb79839bb98431b93b85916aed4a89b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:05.311283 1064 tablet_bootstrap.cc:492] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
W20260810 07:53:05.311261 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cb79839bb98431b93b85916aed4a89b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:05.311795 1064 ts_tablet_manager.cc:1403] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.098s user 0.011s sys 0.017s
I20260810 07:53:05.297444 1874 rpcz_store.cc:276] 0810 07:53:02.407018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.407313 (+ 295us) service_pool.cc:224] Handling call
0810 07:53:04.976734 (+2569421us) ts_tablet_manager.cc:1151] Deleting tablet d7626d30f4574658bb226f2e915ee0de
0810 07:53:04.976797 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.060628 (+ 83831us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.125305 (+ 64677us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.148456 (+ 23151us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2547563}
I20260810 07:53:05.313503 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]) successfully deleted
I20260810 07:53:05.315385 1874 tablet_replica.cc:333] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:05.315469 1064 raft_consensus.cc:359] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.316236 1874 raft_consensus.cc:2243] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:05.316409 1064 raft_consensus.cc:385] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.316696 1874 raft_consensus.cc:2272] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.317625 32469 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 273) took 1132 ms. Trace:
I20260810 07:53:05.318994 1874 ts_tablet_manager.cc:1916] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.317941 32469 rpcz_store.cc:276] 0810 07:53:04.185005 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.331116 (+146111us) service_pool.cc:224] Handling call
0810 07:53:04.451004 (+119888us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c1c6 00007f30cf420034 00007f30cf41ffc2 00007f30cf41fef1 00007f30cf41fe6c 00007f30cf41f122 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:04.453397 (+ 2393us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:04.454817 (+ 1420us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:05.317065 (+862248us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:05.317322 (+ 257us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:05.317566 (+ 244us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.812257 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.812768 (+ 511us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.812801 (+ 33us) write_op.cc:435] Acquired schema lock
0810 07:53:04.812818 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:04.813131 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.813173 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.813202 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:04.813426 (+ 224us) tablet.cc:701] Row locks acquired
0810 07:53:04.813439 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:04.813565 (+ 126us) write_op.cc:270] Start()
0810 07:53:04.813658 (+ 93us) write_op.cc:276] Timestamp: P: 1786348384813491 usec, L: 0
0810 07:53:04.813676 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.814998 (+ 1322us) log.cc:844] Serialized 433 byte log entry
0810 07:53:05.262824 (+447826us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.301143 (+ 38319us) write_op.cc:301] APPLY: starting
0810 07:53:05.301215 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.301334 (+ 119us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.301477 (+ 143us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.302032 (+ 555us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.308891 (+ 6859us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:05.308944 (+ 53us) write_op.cc:312] APPLY: finished
0810 07:53:05.309156 (+ 212us) log.cc:844] Serialized 29 byte log entry
0810 07:53:05.309852 (+ 696us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.309941 (+ 89us) write_op.cc:454] Released schema lock
0810 07:53:05.309981 (+ 40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":273876224,"child_traces":[["op",{"apply.queue_time_us":38083,"mutex_wait_us":33903,"num_ops":2,"prepare.queue_time_us":356253,"prepare.run_cpu_time_us":2419,"prepare.run_wall_time_us":326190,"raft.queue_time_us":244084,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":19,"replication_time_us":449022}]]}
W20260810 07:53:05.326093 32565 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
I20260810 07:53:05.326370 1064 raft_consensus.cc:740] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.326639 32565 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:05.327119 1064 consensus_queue.cc:260] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.327484 2230 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:53:05.328521 2235 consensus_queue.cc:1048] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:05.331030 2235 raft_consensus.cc:493] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.331430 2235 raft_consensus.cc:515] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.335124 2103 raft_consensus.cc:515] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.338773 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:05.339598 877 raft_consensus.cc:3060] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.341854 1064 ts_tablet_manager.cc:1434] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.030s user 0.009s sys 0.000s
I20260810 07:53:05.342855 1064 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.350116 415 raft_consensus.cc:2468] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:05.350955 2251 raft_consensus.cc:2749] T b7bbccbecf884768b3b171d9bd901b9b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.355002 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.355238 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:05.358760 308 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.359862 1064 tablet_bootstrap.cc:654] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:05.359807 308 leader_election.cc:343] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.362695 1071 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:05.368832 2103 leader_election.cc:290] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.371031 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:05.372567 309 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:53:05.373014 309 leader_election.cc:304] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.376050 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_1ac5b694f50044beb1c28f2388c02f4a" } new_table_name: "default.Table_a676bcc87b0c4673adb4eb1808a05542" modify_external_catalogs: true
I20260810 07:53:05.379460 1047 raft_consensus.cc:359] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.380129 1047 raft_consensus.cc:385] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.380360 1874 ts_tablet_manager.cc:1929] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:05.380393 1047 raft_consensus.cc:740] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.380697 1874 log.cc:1199] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/213fa4838fe847de87091cfba97eaedc
I20260810 07:53:05.381393 1874 ts_tablet_manager.cc:1950] T 213fa4838fe847de87091cfba97eaedc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:05.381210 1047 consensus_queue.cc:260] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:05.382804 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 90777.2us
I20260810 07:53:05.383352 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 197) took 2655 ms. Trace:
I20260810 07:53:05.383513 574 leader_election.cc:304] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.383698 1874 rpcz_store.cc:276] 0810 07:53:02.727684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.727825 (+ 141us) service_pool.cc:224] Handling call
0810 07:53:05.314374 (+2586549us) ts_tablet_manager.cc:1151] Deleting tablet 213fa4838fe847de87091cfba97eaedc
0810 07:53:05.314456 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.359868 (+ 45412us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.380228 (+ 20360us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.383293 (+ 3065us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2565015}
I20260810 07:53:05.384974 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 213fa4838fe847de87091cfba97eaedc (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]) successfully deleted
I20260810 07:53:05.391731 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 85f7b2b4d5484e08a5e0f56c7e01f07e
I20260810 07:53:05.393373 1064 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.393656 2235 leader_election.cc:290] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.393847 1064 ts_tablet_manager.cc:1403] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.003s
I20260810 07:53:05.394546 2235 raft_consensus.cc:2749] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.397764 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:04 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:05.397989 2268 log.cc:927] Time spent T 0dbe1a6581374213be7c3f63a7a567a4 P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260810 07:53:05.402346 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:04 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:05.408007 2229 raft_consensus.cc:515] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.398720 1874 tablet_replica.cc:333] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:05.412539 1047 ts_tablet_manager.cc:1434] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.131s user 0.010s sys 0.009s
I20260810 07:53:05.396425 1064 raft_consensus.cc:359] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.413522 1064 raft_consensus.cc:385] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.413798 1064 raft_consensus.cc:740] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.414438 1064 consensus_queue.cc:260] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.416931 2212 raft_consensus.cc:2804] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.426681 1064 ts_tablet_manager.cc:1434] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.033s user 0.004s sys 0.003s
I20260810 07:53:05.427803 1064 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.435227 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet c43aeafc19a240e6991b69249b0516fa from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:05.436419 2263 tablet.cc:1724] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:05.424840 2212 raft_consensus.cc:697] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:05.441741 2212 consensus_queue.cc:237] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.451036 1064 tablet_bootstrap.cc:654] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.455765 1047 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:05.461144 2274 tablet.cc:1724] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:05.475956 2147 raft_consensus.cc:493] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.476441 2147 raft_consensus.cc:515] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.478588 2147 leader_election.cc:290] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.479391 32461 catalog_manager.cc:5719] T 0dbe1a6581374213be7c3f63a7a567a4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:05.480403 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89cdc44a8c64ccd83fd30855798f87b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.481006 676 raft_consensus.cc:2468] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:05.482826 792 leader_election.cc:304] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.486148 2147 raft_consensus.cc:2804] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.486552 2147 raft_consensus.cc:493] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.487016 2147 raft_consensus.cc:3060] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:05.487804 792 leader_election.cc:343] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:05.419869 382 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:05.420711 367 tablet_service.cc:1558] Processing DeleteTablet for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:04 UTC) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:05.495890 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56899.8us
I20260810 07:53:05.460950 2235 raft_consensus.cc:493] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.499006 2235 raft_consensus.cc:515] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.505048 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.505784 676 raft_consensus.cc:2468] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:05.507069 308 leader_election.cc:304] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.508344 2103 raft_consensus.cc:2804] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.509315 2103 raft_consensus.cc:493] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.511687 2103 raft_consensus.cc:3060] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.516373 2274 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 250) took 1217 ms. Trace:
I20260810 07:53:05.516794 2274 rpcz_store.cc:276] 0810 07:53:04.299476 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.299579 (+ 103us) service_pool.cc:224] Handling call
0810 07:53:05.516295 (+1216716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.300619 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:04.301017 (+ 398us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:04.301054 (+ 37us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:04.301123 (+ 69us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:04.301233 (+ 110us) alter_schema_op.cc:128] START. Timestamp: 1786348384301172
0810 07:53:04.301253 (+ 20us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.301739 (+ 486us) log.cc:844] Serialized 173 byte log entry
0810 07:53:04.605992 (+304253us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b18000ee8a0. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22be552c01 00007f22be55f581 00007f22b565dd33 00007f22b5660121 00007f22b56600d9 00007f22b5660069 00007f22b5660031 00007f22b565f32d 00000000005bd8c4 00000000005bd6f9 00007f22b56411ea
0810 07:53:04.635782 (+ 29790us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b18000ee8a0. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22be552c01 00007f22be55f581 00007f22b56407b7 00000000005adb39 00007f22b565cfdb 00007f22b565b2db 00007f22b565ae05 00007f22bc73bf48 00007f22bc73bca2 00007f22bc743efd 00007f22bc73d87a
0810 07:53:04.749388 (+113606us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400190da0. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22bc72c951 00007f22bc741723 00007f22bc751d78 00007f22bc751d19 00007f22bc751ca9 00007f22bc751c71 00007f22bc750f6d 00000000005bd8c4 00000000005bd6f9 00007f22b565ddb5 00007f22b5660121
0810 07:53:05.378889 (+629501us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.461029 (+ 82140us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:05.485918 (+ 24889us) log.cc:844] Serialized 5 byte log entry
0810 07:53:05.511426 (+ 25508us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.512201 (+ 775us) log.cc:844] Serialized 14 byte log entry
0810 07:53:05.516107 (+ 3906us) alter_schema_op.cc:73] Released schema lock
0810 07:53:05.516141 (+ 34us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":81831,"mutex_wait_us":5197,"prepare.queue_time_us":217,"prepare.run_cpu_time_us":105985,"prepare.run_wall_time_us":263847,"raft.queue_time_us":58735,"raft.run_cpu_time_us":133645,"raft.run_wall_time_us":408087,"replication_time_us":1077515,"spinlock_wait_cycles":371605632,"thread_start_us":482654,"threads_started":4,"wal-append.queue_time_us":237473}]]}
I20260810 07:53:05.548002 1600 consensus_queue.cc:1048] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:05.553192 1047 tablet_bootstrap.cc:654] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.558619 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.559237 893 raft_consensus.cc:2468] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:05.560817 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:05.561480 2285 raft_consensus.cc:493] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.561921 2285 raft_consensus.cc:515] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:05.562803 308 leader_election.cc:343] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.564314 2285 leader_election.cc:290] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.566700 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.568146 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.568748 676 raft_consensus.cc:2468] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:05.569931 792 leader_election.cc:304] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.571022 2285 raft_consensus.cc:2804] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.571389 2285 raft_consensus.cc:493] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.571774 2285 raft_consensus.cc:3060] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.576944 2212 raft_consensus.cc:493] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.578995 2212 raft_consensus.cc:515] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.582535 2212 leader_election.cc:290] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.582839 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.583628 893 raft_consensus.cc:2468] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:05.585430 574 leader_election.cc:304] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.594148 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.600950 2287 tablet.cc:1724] T afbac73bbeb44af6b39ab23c5cf79eed P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:05.590988 2212 raft_consensus.cc:2804] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.601948 2212 raft_consensus.cc:493] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.486526 877 raft_consensus.cc:2468] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:05.603839 1874 raft_consensus.cc:2243] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:05.604343 2092 ts_tablet_manager.cc:1929] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:05.604749 2092 log.cc:1199] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/d7626d30f4574658bb226f2e915ee0de
I20260810 07:53:05.604911 309 leader_election.cc:304] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.605491 2235 leader_election.cc:290] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.605952 1874 raft_consensus.cc:2272] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.606292 2235 raft_consensus.cc:2804] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.607962 1874 ts_tablet_manager.cc:1916] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.606928 2235 raft_consensus.cc:697] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:05.610045 2092 ts_tablet_manager.cc:1950] T d7626d30f4574658bb226f2e915ee0de P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:05.609923 2235 consensus_queue.cc:237] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.611812 1071 tablet_bootstrap.cc:654] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.602757 2212 raft_consensus.cc:3060] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.616123 2229 leader_election.cc:290] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:05.534441 32469 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ed74e441397344e09e924fb724ad55d6 consensus state has no leader
I20260810 07:53:05.536027 32468 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 276) took 1001 ms. Trace:
I20260810 07:53:05.617432 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:05.617956 893 raft_consensus.cc:3060] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.617872 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:05.619081 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:05.616947 32468 rpcz_store.cc:276] 0810 07:53:04.534409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.212990 (+678581us) service_pool.cc:224] Handling call
0810 07:53:05.309221 (+ 96231us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c1c6 00007f30cf420034 00007f30cf41ffc2 00007f30cf41fef1 00007f30cf41fe6c 00007f30cf41f122 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:05.311782 (+ 2561us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:05.313526 (+ 1744us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:05.535046 (+221520us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:05.535494 (+ 448us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:05.535925 (+ 431us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:05.376397 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:05.489095 (+112698us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:05.489122 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:53:05.489136 (+ 14us) tablet.cc:662] Decoding operations
0810 07:53:05.489465 (+ 329us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:05.489507 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:05.489531 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:05.489729 (+ 198us) tablet.cc:701] Row locks acquired
0810 07:53:05.489742 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:05.489860 (+ 118us) write_op.cc:270] Start()
0810 07:53:05.489953 (+ 93us) write_op.cc:276] Timestamp: P: 1786348385489817 usec, L: 0
0810 07:53:05.489968 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:53:05.509306 (+ 19338us) log.cc:844] Serialized 433 byte log entry
0810 07:53:05.528131 (+ 18825us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.528445 (+ 314us) write_op.cc:301] APPLY: starting
0810 07:53:05.530392 (+ 1947us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.530534 (+ 142us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.530651 (+ 117us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.531368 (+ 717us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.531550 (+ 182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:05.531615 (+ 65us) write_op.cc:312] APPLY: finished
0810 07:53:05.531840 (+ 225us) log.cc:844] Serialized 29 byte log entry
0810 07:53:05.531990 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.532068 (+ 78us) write_op.cc:454] Released schema lock
0810 07:53:05.532101 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:05.536661 (+ 4560us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.536700 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":212916480,"child_traces":[["op",{"apply.queue_time_us":110,"apply.run_cpu_time_us":2131,"apply.run_wall_time_us":8149,"num_ops":2,"prepare.queue_time_us":61648,"prepare.run_cpu_time_us":2195,"prepare.run_wall_time_us":133200,"raft.queue_time_us":2559,"raft.run_cpu_time_us":706,"raft.run_wall_time_us":1592,"replication_time_us":38063}]]}
W20260810 07:53:05.619761 573 leader_election.cc:343] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.620123 32461 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:05.620393 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_8bed0806196e4e6393b129851ab29404" } modify_external_catalogs: true
W20260810 07:53:05.553802 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet ed74e441397344e09e924fb724ad55d6 (table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ed74e441397344e09e924fb724ad55d6 consensus state has no leader
I20260810 07:53:05.624384 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_8bed0806196e4e6393b129851ab29404" } modify_external_catalogs: true
I20260810 07:53:05.625748 2103 raft_consensus.cc:515] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.626119 893 raft_consensus.cc:2468] T f8415b6376a740388487f7ae4aebea52 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:05.626981 407 raft_consensus.cc:2393] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:05.627307 309 leader_election.cc:304] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.622201 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_72d628558a2d41caa3fb7a5baed5ba51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:05.630812 2147 raft_consensus.cc:515] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:05.631907 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72d628558a2d41caa3fb7a5baed5ba51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:05.634637 1064 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.635070 1064 ts_tablet_manager.cc:1403] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.207s user 0.020s sys 0.048s
I20260810 07:53:05.637694 2285 raft_consensus.cc:515] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.638736 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 221) took 2608 ms. Trace:
I20260810 07:53:05.639696 2285 leader_election.cc:290] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.640966 1064 raft_consensus.cc:359] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.641179 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:05.641583 1064 raft_consensus.cc:385] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.641698 676 raft_consensus.cc:3060] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.641839 1064 raft_consensus.cc:740] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.639706 2092 rpcz_store.cc:276] 0810 07:53:03.030262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.030398 (+ 136us) service_pool.cc:224] Handling call
0810 07:53:04.953707 (+1923309us) ts_tablet_manager.cc:1151] Deleting tablet d7626d30f4574658bb226f2e915ee0de
0810 07:53:04.953783 (+ 76us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.562523 (+608740us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.604198 (+ 41675us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.638657 (+ 34459us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1922614}
I20260810 07:53:05.642465 1064 consensus_queue.cc:260] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.643466 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 222) took 2612 ms. Trace:
I20260810 07:53:05.643745 2092 rpcz_store.cc:276] 0810 07:53:03.031358 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.031450 (+ 92us) service_pool.cc:224] Handling call
0810 07:53:05.643094 (+2611644us) ts_tablet_manager.cc:1151] Deleting tablet 213fa4838fe847de87091cfba97eaedc
0810 07:53:05.643137 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.643414 (+ 277us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2611151}
I20260810 07:53:05.648053 2103 leader_election.cc:290] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.648591 2103 raft_consensus.cc:2804] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.649111 2103 raft_consensus.cc:697] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:05.649525 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:05.650393 893 raft_consensus.cc:2393] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:05.651072 676 raft_consensus.cc:2468] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:05.652165 792 leader_election.cc:304] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.653450 2212 raft_consensus.cc:515] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.653759 2147 leader_election.cc:290] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.656975 2212 leader_election.cc:290] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.656991 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c43aeafc19a240e6991b69249b0516fa
I20260810 07:53:05.658519 32464 catalog_manager.cc:6807] ReplaceTablet: tablet c43aeafc19a240e6991b69249b0516fa deleted and replaced by tablet a6077111ffb2411892b62d806ea3c85b
I20260810 07:53:05.655154 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 225) took 2531 ms. Trace:
I20260810 07:53:05.660889 2092 rpcz_store.cc:276] 0810 07:53:03.123985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.124101 (+ 116us) service_pool.cc:224] Handling call
0810 07:53:05.644534 (+2520433us) ts_tablet_manager.cc:1151] Deleting tablet 7000d25f3bfe4c99be54a0f4789cc96c
0810 07:53:05.644570 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.655088 (+ 10518us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2519563}
I20260810 07:53:05.662154 859 tablet_service.cc:1558] Processing DeleteTablet for tablet c43aeafc19a240e6991b69249b0516fa with delete_type TABLET_DATA_DELETED (replaced by tablet a6077111ffb2411892b62d806ea3c85b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:05.662318 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 228) took 2277 ms. Trace:
I20260810 07:53:05.662640 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_29e9be4c452b438e82564903ce411e53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:05.663266 2092 rpcz_store.cc:276] 0810 07:53:03.384908 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.385434 (+ 526us) service_pool.cc:224] Handling call
0810 07:53:05.662010 (+2276576us) ts_tablet_manager.cc:1151] Deleting tablet 9ae4a28e89714c65b7551f36d1dc78b4
0810 07:53:05.662055 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.662243 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2275903}
I20260810 07:53:05.663826 1064 ts_tablet_manager.cc:1434] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
W20260810 07:53:05.664898 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29e9be4c452b438e82564903ce411e53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:05.661091 2103 consensus_queue.cc:237] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.665247 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 229) took 2274 ms. Trace:
I20260810 07:53:05.666471 2092 rpcz_store.cc:276] 0810 07:53:03.390901 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.394731 (+ 3830us) service_pool.cc:224] Handling call
0810 07:53:05.664970 (+2270239us) ts_tablet_manager.cc:1151] Deleting tablet 449359c8e9fa4756ba149f3dce6e0ce7
0810 07:53:05.665016 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.665196 (+ 180us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2269488}
I20260810 07:53:05.668079 2288 raft_consensus.cc:2804] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.668337 2092 tablet_replica.cc:333] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:05.669196 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89cdc44a8c64ccd83fd30855798f87b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:05.669821 678 raft_consensus.cc:3060] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.670703 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:05.671423 893 raft_consensus.cc:3060] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.676635 1874 ts_tablet_manager.cc:1929] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:05.677095 1874 log.cc:1199] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:05.677744 1874 ts_tablet_manager.cc:1950] T 7000d25f3bfe4c99be54a0f4789cc96c P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:05.678010 382 tablet_service.cc:1558] Processing DeleteTablet for tablet c43aeafc19a240e6991b69249b0516fa with delete_type TABLET_DATA_DELETED (replaced by tablet a6077111ffb2411892b62d806ea3c85b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:05.678622 678 raft_consensus.cc:2468] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:05.678769 1064 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.679876 792 leader_election.cc:304] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.680737 2285 raft_consensus.cc:2804] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.681354 2285 raft_consensus.cc:697] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:05.682394 2285 consensus_queue.cc:237] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.685698 1047 tablet_bootstrap.cc:492] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:05.695932 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 199) took 2814 ms. Trace:
I20260810 07:53:05.696700 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet d7626d30f4574658bb226f2e915ee0de (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]) successfully deleted
W20260810 07:53:05.698223 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 213fa4838fe847de87091cfba97eaedc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 213fa4838fe847de87091cfba97eaedc already in progress: creating tablet
W20260810 07:53:05.699314 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7000d25f3bfe4c99be54a0f4789cc96c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7000d25f3bfe4c99be54a0f4789cc96c already in progress: creating tablet
W20260810 07:53:05.700397 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 9ae4a28e89714c65b7551f36d1dc78b4 because the tablet was not found. No further retry: Not found: Tablet not found: 9ae4a28e89714c65b7551f36d1dc78b4
W20260810 07:53:05.701428 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 449359c8e9fa4756ba149f3dce6e0ce7 because the tablet was not found. No further retry: Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:05.701983 1874 rpcz_store.cc:276] 0810 07:53:02.881680 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:02.884036 (+ 2356us) service_pool.cc:224] Handling call
0810 07:53:05.395010 (+2510974us) ts_tablet_manager.cc:1151] Deleting tablet 7000d25f3bfe4c99be54a0f4789cc96c
0810 07:53:05.398578 (+ 3568us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.605723 (+207145us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000e47d4. stack: 00007f49ea319cf3 00007f49ea319bf0 00007f49e9ec7574 0000000000516de4 0000000000516d79 00007f49f141a951 00007f49f14b9742 00007f49f18f9d2c 00007f49f256141a 00007f49f25920e0 00007f49f257864d 00007f49f25785f9 00007f49f2578589 00007f49f2578551 00007f49f257777d 00000000005bd8c4
0810 07:53:05.615655 (+ 9932us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.676482 (+ 60827us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.695869 (+ 19387us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24095,"spinlock_wait_cycles":408239488,"tablet-delete.queue_time_us":2510354}
I20260810 07:53:05.708179 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 7000d25f3bfe4c99be54a0f4789cc96c (table default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]) successfully deleted
I20260810 07:53:05.710362 1064 tablet_bootstrap.cc:654] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.600987 2283 tablet_replica.cc:333] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:05.715101 893 raft_consensus.cc:2468] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:05.711164 1874 tablet_replica.cc:333] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:05.722867 1874 raft_consensus.cc:2243] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:05.723320 2283 raft_consensus.cc:2243] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:05.724035 2283 raft_consensus.cc:2272] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.726354 2283 ts_tablet_manager.cc:1916] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.686156 1047 ts_tablet_manager.cc:1403] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.231s user 0.007s sys 0.003s
I20260810 07:53:05.731040 1874 raft_consensus.cc:2272] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.728842 2288 raft_consensus.cc:697] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
W20260810 07:53:05.695299 574 consensus_peers.cc:597] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:05.669176 2092 raft_consensus.cc:2243] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:05.732138 2288 consensus_queue.cc:237] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.732874 574 leader_election.cc:304] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.733193 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:05.731777 623 tablet_service.cc:1558] Processing DeleteTablet for tablet c43aeafc19a240e6991b69249b0516fa with delete_type TABLET_DATA_DELETED (replaced by tablet a6077111ffb2411892b62d806ea3c85b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:05.735579 676 raft_consensus.cc:2393] T 10341e269a934612bb0cf6ef29818a39 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:05.736131 1071 tablet_bootstrap.cc:492] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:05.736636 1071 ts_tablet_manager.cc:1403] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.409s user 0.007s sys 0.003s
I20260810 07:53:05.737196 2092 raft_consensus.cc:2272] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.738477 1874 ts_tablet_manager.cc:1916] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.739200 1071 raft_consensus.cc:359] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.739804 2092 ts_tablet_manager.cc:1916] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.739835 1071 raft_consensus.cc:385] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.740103 1071 raft_consensus.cc:740] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.742360 1671 raft_consensus.cc:2804] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:05.742794 1671 raft_consensus.cc:697] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:05.743513 1671 consensus_queue.cc:237] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.749758 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:05.752503 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e0a84839427099a9da06e2b970ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:05.758332 32468 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_ae927dcca7ec42b99eee31143de3bf0b" } modify_external_catalogs: true
I20260810 07:53:05.759014 32468 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_ae927dcca7ec42b99eee31143de3bf0b" } modify_external_catalogs: true
I20260810 07:53:05.761428 308 leader_election.cc:304] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.763271 2103 raft_consensus.cc:2749] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.764036 1064 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.764570 1064 ts_tablet_manager.cc:1403] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.086s user 0.005s sys 0.010s
I20260810 07:53:05.766906 623 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e0a84839427099a9da06e2b970ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:05.767434 1064 raft_consensus.cc:359] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.768164 1064 raft_consensus.cc:385] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.772927 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e0a84839427099a9da06e2b970ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:05.768474 1064 raft_consensus.cc:740] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.781384 1064 consensus_queue.cc:260] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.785169 1064 ts_tablet_manager.cc:1434] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260810 07:53:05.786198 1064 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:05.797252 1874 ts_tablet_manager.cc:1929] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:05.731729 1047 raft_consensus.cc:359] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.806051 1047 raft_consensus.cc:385] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.806344 1047 raft_consensus.cc:740] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.806951 1047 consensus_queue.cc:260] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.809444 1047 ts_tablet_manager.cc:1434] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.080s user 0.003s sys 0.003s
I20260810 07:53:05.814335 1071 consensus_queue.cc:260] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.815207 2235 raft_consensus.cc:493] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.802830 2288 raft_consensus.cc:493] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.815717 2235 raft_consensus.cc:515] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.815834 2288 raft_consensus.cc:515] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.818226 2288 leader_election.cc:290] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.820573 32469 master_service.cc:960] ReplaceTablet: received request to replace tablet e1fa9200140a41dfbd886442ae336988 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:05.821041 499 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
I20260810 07:53:05.822362 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:05.823968 792 leader_election.cc:343] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.824867 1874 log.cc:1199] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/54a3bd8369634e4bbfdf8920454357bc
I20260810 07:53:05.829051 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.830106 2235 leader_election.cc:290] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:05.832691 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:05.833624 415 raft_consensus.cc:2393] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:05.829594 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.850294 893 raft_consensus.cc:2393] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:05.851745 309 leader_election.cc:304] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:05.835235 792 leader_election.cc:343] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:05.852779 2229 raft_consensus.cc:3060] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.856711 1071 ts_tablet_manager.cc:1434] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.120s user 0.005s sys 0.001s
I20260810 07:53:05.858296 1064 tablet_bootstrap.cc:654] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.859992 2092 ts_tablet_manager.cc:1929] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:05.859146 1874 ts_tablet_manager.cc:1950] T 54a3bd8369634e4bbfdf8920454357bc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:05.866844 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 203) took 2767 ms. Trace:
I20260810 07:53:05.867256 1874 rpcz_store.cc:276] 0810 07:53:03.099369 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.099523 (+ 154us) service_pool.cc:224] Handling call
0810 07:53:05.710962 (+2611439us) ts_tablet_manager.cc:1151] Deleting tablet 54a3bd8369634e4bbfdf8920454357bc
0810 07:53:05.711030 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.770859 (+ 59829us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.797107 (+ 26248us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.866779 (+ 69672us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2610748}
I20260810 07:53:05.868762 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 54a3bd8369634e4bbfdf8920454357bc (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]) successfully deleted
I20260810 07:53:05.869967 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eeabaae541548afab4a3ff5ab388035" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:05.871397 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.869948 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 204) took 2767 ms. Trace:
I20260810 07:53:05.879333 1874 rpcz_store.cc:276] 0810 07:53:03.102706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.102837 (+ 131us) service_pool.cc:224] Handling call
0810 07:53:05.869640 (+2766803us) ts_tablet_manager.cc:1151] Deleting tablet 9ae4a28e89714c65b7551f36d1dc78b4
0810 07:53:05.869692 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.869894 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2766187}
W20260810 07:53:05.880949 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 9ae4a28e89714c65b7551f36d1dc78b4 because the tablet was not found. No further retry: Not found: Tablet not found: 9ae4a28e89714c65b7551f36d1dc78b4
I20260810 07:53:05.880997 2092 log.cc:1199] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:05.881937 2092 ts_tablet_manager.cc:1950] T c1faed468bfa4d1ba5d92c0b9b194781 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:05.892915 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 230) took 2496 ms. Trace:
I20260810 07:53:05.893370 2229 raft_consensus.cc:2749] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:05.895090 792 leader_election.cc:343] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.895566 792 leader_election.cc:304] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.897011 2288 raft_consensus.cc:2749] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.897500 2283 ts_tablet_manager.cc:1929] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:05.897917 2283 log.cc:1199] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:05.898833 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 205) took 2792 ms. Trace:
I20260810 07:53:05.898948 2283 ts_tablet_manager.cc:1950] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:05.899986 1874 rpcz_store.cc:276] 0810 07:53:03.105993 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.106153 (+ 160us) service_pool.cc:224] Handling call
0810 07:53:05.886933 (+2780780us) ts_tablet_manager.cc:1151] Deleting tablet cf45a33485e9463d89b5ce8affa0bd22
0810 07:53:05.886996 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.887235 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2780010}
W20260810 07:53:05.901681 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because the tablet was not found. No further retry: Not found: Tablet not found: cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:53:05.901894 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]) successfully deleted
W20260810 07:53:05.902827 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f7b2b4d5484e08a5e0f56c7e01f07e already in progress: creating tablet
I20260810 07:53:05.900832 412 raft_consensus.cc:2468] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:05.904860 2092 rpcz_store.cc:276] 0810 07:53:03.396097 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.399127 (+ 3030us) service_pool.cc:224] Handling call
0810 07:53:05.668131 (+2269004us) ts_tablet_manager.cc:1151] Deleting tablet c1faed468bfa4d1ba5d92c0b9b194781
0810 07:53:05.668193 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.814056 (+145863us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.859848 (+ 45792us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.892846 (+ 32998us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3924,"tablet-delete.queue_time_us":2268408}
I20260810 07:53:05.888072 1047 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:05.907186 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet c1faed468bfa4d1ba5d92c0b9b194781 (table default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]) successfully deleted
I20260810 07:53:05.909319 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 231) took 2509 ms. Trace:
I20260810 07:53:05.910750 2092 rpcz_store.cc:276] 0810 07:53:03.400218 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.400329 (+ 111us) service_pool.cc:224] Handling call
0810 07:53:05.909007 (+2508678us) ts_tablet_manager.cc:1151] Deleting tablet cf45a33485e9463d89b5ce8affa0bd22
0810 07:53:05.909054 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.909250 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2507986}
I20260810 07:53:05.903874 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 206) took 2795 ms. Trace:
I20260810 07:53:05.912937 1874 rpcz_store.cc:276] 0810 07:53:03.108541 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.108738 (+ 197us) service_pool.cc:224] Handling call
0810 07:53:05.902612 (+2793874us) ts_tablet_manager.cc:1151] Deleting tablet c1faed468bfa4d1ba5d92c0b9b194781
0810 07:53:05.902675 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.903807 (+ 1132us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2793295}
I20260810 07:53:05.914429 2092 tablet_replica.cc:333] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:53:05.914430 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because the tablet was not found. No further retry: Not found: Tablet not found: c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:05.915679 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 209) took 2737 ms. Trace:
W20260810 07:53:05.916326 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet cf45a33485e9463d89b5ce8affa0bd22 because the tablet was not found. No further retry: Not found: Tablet not found: cf45a33485e9463d89b5ce8affa0bd22
I20260810 07:53:05.917403 1071 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:05.916102 1874 rpcz_store.cc:276] 0810 07:53:03.178067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.178250 (+ 183us) service_pool.cc:224] Handling call
0810 07:53:05.915284 (+2737034us) ts_tablet_manager.cc:1151] Deleting tablet 08520a2ac0f64c3abd534a32100839a4
0810 07:53:05.915354 (+ 70us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.915603 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2736328}
W20260810 07:53:05.919847 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 08520a2ac0f64c3abd534a32100839a4 because the tablet was not found. No further retry: Not found: Tablet not found: 08520a2ac0f64c3abd534a32100839a4
I20260810 07:53:05.921020 1874 tablet_replica.cc:333] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:05.930346 1874 raft_consensus.cc:2243] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Raft consensus shutting down.
W20260810 07:53:05.930635 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet c43aeafc19a240e6991b69249b0516fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c43aeafc19a240e6991b69249b0516fa already in progress: creating tablet
I20260810 07:53:05.931265 1874 pending_rounds.cc:70] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Trying to abort 1 pending ops.
I20260810 07:53:05.921553 2092 raft_consensus.cc:2243] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:05.931559 1874 pending_rounds.cc:77] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316882987769892864 op_type: NO_OP noop_request { }
I20260810 07:53:05.931847 2092 raft_consensus.cc:2272] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.932289 1874 raft_consensus.cc:2889] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:53:05.933194 2285 raft_consensus.cc:493] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.933622 2285 raft_consensus.cc:515] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.934010 2092 ts_tablet_manager.cc:1916] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.935581 2285 leader_election.cc:290] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.937655 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:05.939447 792 leader_election.cc:343] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.939641 1071 tablet_bootstrap.cc:654] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:05.932752 1874 raft_consensus.cc:2272] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:05.949065 1874 ts_tablet_manager.cc:1916] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:05.952370 1064 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:05.940483 2235 raft_consensus.cc:493] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.952859 32469 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet e1fa9200140a41dfbd886442ae336988
I20260810 07:53:05.953009 2235 raft_consensus.cc:515] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:05.954947 382 tablet_service.cc:1558] Processing DeleteTablet for tablet e1fa9200140a41dfbd886442ae336988 with delete_type TABLET_DATA_DELETED (replaced by tablet e0a74b22a6864180956e763d2b6b6de6) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:05.956080 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc12205a097c496cba5b777b98f27914" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:05.956620 676 raft_consensus.cc:2468] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:05.959483 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc12205a097c496cba5b777b98f27914" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:05.960116 893 raft_consensus.cc:2468] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:05.960057 308 leader_election.cc:304] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:05.967198 1874 ts_tablet_manager.cc:1929] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:05.967152 623 tablet_service.cc:1558] Processing DeleteTablet for tablet e1fa9200140a41dfbd886442ae336988 with delete_type TABLET_DATA_DELETED (replaced by tablet e0a74b22a6864180956e763d2b6b6de6) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:05.967357 1064 ts_tablet_manager.cc:1403] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.181s user 0.012s sys 0.003s
I20260810 07:53:05.967547 1874 log.cc:1199] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:05.968290 1874 ts_tablet_manager.cc:1950] T a89cdc44a8c64ccd83fd30855798f87b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:05.969296 2229 raft_consensus.cc:2804] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:05.970124 859 tablet_service.cc:1558] Processing DeleteTablet for tablet e1fa9200140a41dfbd886442ae336988 with delete_type TABLET_DATA_DELETED (replaced by tablet e0a74b22a6864180956e763d2b6b6de6) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:05.969767 2229 raft_consensus.cc:493] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:05.970296 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 214) took 2513 ms. Trace:
I20260810 07:53:05.970830 32469 catalog_manager.cc:6807] ReplaceTablet: tablet e1fa9200140a41dfbd886442ae336988 deleted and replaced by tablet e0a74b22a6864180956e763d2b6b6de6
I20260810 07:53:05.970600 1874 rpcz_store.cc:276] 0810 07:53:03.457028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.457191 (+ 163us) service_pool.cc:224] Handling call
0810 07:53:05.920847 (+2463656us) ts_tablet_manager.cc:1151] Deleting tablet a89cdc44a8c64ccd83fd30855798f87b
0810 07:53:05.920900 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.957825 (+ 36925us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.967079 (+ 9254us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:05.970241 (+ 3162us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2462902}
I20260810 07:53:05.971948 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet a89cdc44a8c64ccd83fd30855798f87b (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
I20260810 07:53:05.970558 2229 raft_consensus.cc:3060] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:05.974037 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_80bd87f8139444628cd4688972911389"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:05.976821 2235 leader_election.cc:290] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:05.975716 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80bd87f8139444628cd4688972911389 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:05.978075 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:05.979494 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 226) took 1673 ms. Trace:
W20260810 07:53:05.979523 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 8d92e0a84839427099a9da06e2b970ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d92e0a84839427099a9da06e2b970ce already in progress: creating tablet
W20260810 07:53:05.979907 792 leader_election.cc:343] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.978926 1064 raft_consensus.cc:359] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.980324 792 leader_election.cc:304] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.980584 1064 raft_consensus.cc:385] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:05.980970 1064 raft_consensus.cc:740] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:05.981333 2285 raft_consensus.cc:2749] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:05.981743 1064 consensus_queue.cc:260] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:05.982511 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet e1fa9200140a41dfbd886442ae336988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1fa9200140a41dfbd886442ae336988 already in progress: creating tablet
I20260810 07:53:05.984221 1064 ts_tablet_manager.cc:1434] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260810 07:53:05.979837 1874 rpcz_store.cc:276] 0810 07:53:04.307591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.341407 (+ 33816us) service_pool.cc:224] Handling call
0810 07:53:05.971538 (+1630131us) ts_tablet_manager.cc:1151] Deleting tablet a4863f3f03104d31ad4dafda6f33c9af
0810 07:53:05.971588 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.979419 (+ 7831us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1629087}
W20260810 07:53:05.985109 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet a4863f3f03104d31ad4dafda6f33c9af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4863f3f03104d31ad4dafda6f33c9af already in progress: creating tablet
I20260810 07:53:05.988343 2285 raft_consensus.cc:493] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:05.988804 2285 raft_consensus.cc:515] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:05.990708 2285 leader_election.cc:290] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:05.992614 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:05.994673 792 leader_election.cc:343] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.995484 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:05.996856 792 leader_election.cc:343] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:05.997282 792 leader_election.cc:304] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:05.998252 2285 raft_consensus.cc:2749] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:06.002851 1064 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:06.004845 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 68931.7us
I20260810 07:53:06.007658 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 223) took 2219 ms. Trace:
I20260810 07:53:06.008033 1874 rpcz_store.cc:276] 0810 07:53:03.787923 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.788071 (+ 148us) service_pool.cc:224] Handling call
0810 07:53:05.986613 (+2198542us) ts_tablet_manager.cc:1151] Deleting tablet fe8b4892a53742f48dd43397c4f8e88f
0810 07:53:06.007297 (+ 20684us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.007602 (+ 305us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1604645}
W20260810 07:53:06.017861 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260810 07:53:06.026438 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 227) took 1439 ms. Trace:
I20260810 07:53:06.026834 1874 rpcz_store.cc:276] 0810 07:53:04.587032 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.628996 (+ 41964us) service_pool.cc:224] Handling call
0810 07:53:06.016949 (+1387953us) ts_tablet_manager.cc:1151] Deleting tablet 8d717f39a6704199abc34b220d795e48
0810 07:53:06.021761 (+ 4812us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.026376 (+ 4615us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1379513}
I20260810 07:53:06.027411 2229 raft_consensus.cc:515] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.027959 1874 tablet_replica.cc:333] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:06.028777 1874 raft_consensus.cc:2243] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:06.029276 1874 raft_consensus.cc:2272] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:06.032132 1874 ts_tablet_manager.cc:1916] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:06.032871 2285 raft_consensus.cc:493] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.033377 2285 raft_consensus.cc:515] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.035349 2285 leader_election.cc:290] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:06.037566 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:06.038556 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:06.012466 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet fe8b4892a53742f48dd43397c4f8e88f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8b4892a53742f48dd43397c4f8e88f already in progress: creating tablet
I20260810 07:53:06.016559 2092 ts_tablet_manager.cc:1929] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:06.039832 2092 log.cc:1199] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/2c171d3358e44ae2962ab3c4c377b0f1
W20260810 07:53:06.040016 792 leader_election.cc:343] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.040539 2092 ts_tablet_manager.cc:1950] T 2c171d3358e44ae2962ab3c4c377b0f1 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:06.040977 792 leader_election.cc:343] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.041334 792 leader_election.cc:304] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:06.042255 2285 raft_consensus.cc:2749] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:06.040377 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 8d717f39a6704199abc34b220d795e48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d717f39a6704199abc34b220d795e48 already in progress: creating tablet
I20260810 07:53:06.051170 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc12205a097c496cba5b777b98f27914" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:06.051723 676 raft_consensus.cc:3060] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.051968 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc12205a097c496cba5b777b98f27914" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:06.052539 893 raft_consensus.cc:3060] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.053576 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 232) took 2652 ms. Trace:
I20260810 07:53:06.053967 2092 rpcz_store.cc:276] 0810 07:53:03.401365 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.404892 (+ 3527us) service_pool.cc:224] Handling call
0810 07:53:05.914229 (+2509337us) ts_tablet_manager.cc:1151] Deleting tablet 2c171d3358e44ae2962ab3c4c377b0f1
0810 07:53:05.914287 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:05.985480 (+ 71193us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.016418 (+ 30938us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.053494 (+ 37076us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2508791}
I20260810 07:53:06.055632 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 233) took 2649 ms. Trace:
I20260810 07:53:06.056492 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 2c171d3358e44ae2962ab3c4c377b0f1 (table default.Table_c82ef7c9085a43d8aa2b06a7799dcab9 [id=43a67562d28b442a86908e5e13f18832]) successfully deleted
I20260810 07:53:06.059224 893 raft_consensus.cc:2468] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:06.060372 309 leader_election.cc:304] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:06.061196 2235 raft_consensus.cc:2804] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:06.061661 2235 raft_consensus.cc:697] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:06.062479 2235 consensus_queue.cc:237] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.070472 2285 raft_consensus.cc:493] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.070958 2285 raft_consensus.cc:515] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.055979 2092 rpcz_store.cc:276] 0810 07:53:03.405993 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.406095 (+ 102us) service_pool.cc:224] Handling call
0810 07:53:06.055317 (+2649222us) ts_tablet_manager.cc:1151] Deleting tablet 08520a2ac0f64c3abd534a32100839a4
0810 07:53:06.055374 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.055577 (+ 203us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2648675}
I20260810 07:53:06.073591 2092 tablet_replica.cc:333] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:06.073809 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:06.074640 2092 raft_consensus.cc:2243] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:06.075142 2092 raft_consensus.cc:2272] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:53:06.075222 792 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:06.075399 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 08520a2ac0f64c3abd534a32100839a4 because the tablet was not found. No further retry: Not found: Tablet not found: 08520a2ac0f64c3abd534a32100839a4
I20260810 07:53:06.075984 2285 leader_election.cc:290] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:06.077132 2092 ts_tablet_manager.cc:1916] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:06.077355 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:06.079080 792 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.079519 792 leader_election.cc:304] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:06.080502 2285 raft_consensus.cc:2749] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:06.081027 676 raft_consensus.cc:2468] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:06.083940 2229 leader_election.cc:290] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:06.091951 1874 ts_tablet_manager.cc:1929] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:06.092422 1874 log.cc:1199] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:06.093230 1874 ts_tablet_manager.cc:1950] T acc89d15b4d544b7bc8abc12a4c6d3e5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:06.095654 1874 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 228) took 1002 ms. Trace:
I20260810 07:53:06.096010 1874 rpcz_store.cc:276] 0810 07:53:05.093755 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.093900 (+ 145us) service_pool.cc:224] Handling call
0810 07:53:06.027757 (+933857us) ts_tablet_manager.cc:1151] Deleting tablet acc89d15b4d544b7bc8abc12a4c6d3e5
0810 07:53:06.027810 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.066923 (+ 39113us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.091792 (+ 24869us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.095574 (+ 3782us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":933230}
I20260810 07:53:06.097553 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]) successfully deleted
I20260810 07:53:06.099207 1071 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:06.099735 1071 ts_tablet_manager.cc:1403] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.183s user 0.007s sys 0.007s
W20260810 07:53:06.101411 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f7b2b4d5484e08a5e0f56c7e01f07e already in progress: creating tablet
I20260810 07:53:06.102515 1064 tablet_bootstrap.cc:654] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:06.105576 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c43aeafc19a240e6991b69249b0516fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c43aeafc19a240e6991b69249b0516fa already in progress: creating tablet
I20260810 07:53:06.107275 961 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
W20260810 07:53:06.107539 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 8d92e0a84839427099a9da06e2b970ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d92e0a84839427099a9da06e2b970ce already in progress: creating tablet
I20260810 07:53:06.107662 32469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_ff19276d47ee41718e958f3f15343ac9" } new_table_name: "default.Table_c849f5bbe83b49f1a5b41b6e28baacac" modify_external_catalogs: true
I20260810 07:53:06.113116 2092 ts_tablet_manager.cc:1929] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:06.113564 2092 log.cc:1199] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/a89cdc44a8c64ccd83fd30855798f87b
I20260810 07:53:06.114255 2092 ts_tablet_manager.cc:1950] T a89cdc44a8c64ccd83fd30855798f87b P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:06.116334 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 242) took 2141 ms. Trace:
I20260810 07:53:06.116678 2092 rpcz_store.cc:276] 0810 07:53:03.974558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.975271 (+ 713us) service_pool.cc:224] Handling call
0810 07:53:06.073236 (+2097965us) ts_tablet_manager.cc:1151] Deleting tablet a89cdc44a8c64ccd83fd30855798f87b
0810 07:53:06.073287 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.086683 (+ 13396us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.112968 (+ 26285us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.116267 (+ 3299us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097304}
I20260810 07:53:06.118188 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet a89cdc44a8c64ccd83fd30855798f87b (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]) successfully deleted
W20260810 07:53:06.120271 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet e1fa9200140a41dfbd886442ae336988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1fa9200140a41dfbd886442ae336988 already in progress: creating tablet
I20260810 07:53:06.121802 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 256) took 1795 ms. Trace:
I20260810 07:53:06.122267 2092 rpcz_store.cc:276] 0810 07:53:04.326515 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.328358 (+ 1843us) service_pool.cc:224] Handling call
0810 07:53:06.121418 (+1793060us) ts_tablet_manager.cc:1151] Deleting tablet a4863f3f03104d31ad4dafda6f33c9af
0810 07:53:06.121476 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.121744 (+ 268us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792456}
W20260810 07:53:06.123736 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet a4863f3f03104d31ad4dafda6f33c9af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4863f3f03104d31ad4dafda6f33c9af already in progress: creating tablet
I20260810 07:53:06.124950 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 253) took 1799 ms. Trace:
I20260810 07:53:06.125337 2092 rpcz_store.cc:276] 0810 07:53:04.325239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.326260 (+ 1021us) service_pool.cc:224] Handling call
0810 07:53:06.124560 (+1798300us) ts_tablet_manager.cc:1151] Deleting tablet fe8b4892a53742f48dd43397c4f8e88f
0810 07:53:06.124607 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.124898 (+ 291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792292}
W20260810 07:53:06.126772 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet fe8b4892a53742f48dd43397c4f8e88f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8b4892a53742f48dd43397c4f8e88f already in progress: creating tablet
I20260810 07:53:06.127198 1071 raft_consensus.cc:359] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.127949 1071 raft_consensus.cc:385] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.128255 1071 raft_consensus.cc:740] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.128278 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 257) took 1561 ms. Trace:
I20260810 07:53:06.128715 2092 rpcz_store.cc:276] 0810 07:53:04.569054 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:04.578438 (+ 9384us) service_pool.cc:224] Handling call
0810 07:53:06.127717 (+1549279us) ts_tablet_manager.cc:1151] Deleting tablet 8d717f39a6704199abc34b220d795e48
0810 07:53:06.127763 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.128172 (+ 409us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1548521}
W20260810 07:53:06.130196 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 8d717f39a6704199abc34b220d795e48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d717f39a6704199abc34b220d795e48 already in progress: creating tablet
I20260810 07:53:06.131211 2092 tablet_replica.cc:333] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:06.132124 2092 raft_consensus.cc:2243] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:06.133152 2092 raft_consensus.cc:2272] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:06.135268 2092 ts_tablet_manager.cc:1916] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:06.154304 623 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91af48fe6a4bd986b5266f09fad6b2 (DEFAULT_TABLE table=default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.155634 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91af48fe6a4bd986b5266f09fad6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.168474 382 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91af48fe6a4bd986b5266f09fad6b2 (DEFAULT_TABLE table=default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.169713 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91af48fe6a4bd986b5266f09fad6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.169691 859 tablet_service.cc:1511] Processing CreateTablet for tablet 2a91af48fe6a4bd986b5266f09fad6b2 (DEFAULT_TABLE table=default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.170962 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a91af48fe6a4bd986b5266f09fad6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.167708 32462 catalog_manager.cc:5719] T ab2e7619a25b437c8ee2c817255ca638 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:06.180701 1071 consensus_queue.cc:260] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.182655 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:06.193703 1071 ts_tablet_manager.cc:1434] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.094s user 0.009s sys 0.000s
I20260810 07:53:06.202397 2092 ts_tablet_manager.cc:1929] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:06.202785 2092 log.cc:1199] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:06.203526 2092 ts_tablet_manager.cc:1950] T acc89d15b4d544b7bc8abc12a4c6d3e5 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:06.204223 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:06.205322 32462 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:06.205725 2092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 258) took 1094 ms. Trace:
I20260810 07:53:06.206148 619 tablet_service.cc:1511] Processing CreateTablet for tablet 69f1405e9526433a8fd727ed237f8c84 (DEFAULT_TABLE table=default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.206038 2092 rpcz_store.cc:276] 0810 07:53:05.111187 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.111441 (+ 254us) service_pool.cc:224] Handling call
0810 07:53:06.131043 (+1019602us) ts_tablet_manager.cc:1151] Deleting tablet acc89d15b4d544b7bc8abc12a4c6d3e5
0810 07:53:06.131087 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:06.143576 (+ 12489us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.202271 (+ 58695us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:06.205670 (+ 3399us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":975346}
I20260810 07:53:06.207103 1071 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:06.207330 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1405e9526433a8fd727ed237f8c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.209889 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]) successfully deleted
W20260810 07:53:06.210650 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 85f7b2b4d5484e08a5e0f56c7e01f07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85f7b2b4d5484e08a5e0f56c7e01f07e already in progress: creating tablet
W20260810 07:53:06.211257 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet c43aeafc19a240e6991b69249b0516fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c43aeafc19a240e6991b69249b0516fa already in progress: creating tablet
W20260810 07:53:06.221586 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 8d92e0a84839427099a9da06e2b970ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d92e0a84839427099a9da06e2b970ce already in progress: creating tablet
W20260810 07:53:06.222486 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet e1fa9200140a41dfbd886442ae336988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1fa9200140a41dfbd886442ae336988 already in progress: creating tablet
I20260810 07:53:06.234002 367 tablet_service.cc:1511] Processing CreateTablet for tablet 69f1405e9526433a8fd727ed237f8c84 (DEFAULT_TABLE table=default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.234472 866 tablet_service.cc:1511] Processing CreateTablet for tablet 69f1405e9526433a8fd727ed237f8c84 (DEFAULT_TABLE table=default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.235574 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1405e9526433a8fd727ed237f8c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.235888 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1405e9526433a8fd727ed237f8c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.241710 1071 tablet_bootstrap.cc:654] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:06.245568 574 consensus_peers.cc:597] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54a3bd8369634e4bbfdf8920454357bc. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:06.257985 862 tablet_service.cc:1511] Processing CreateTablet for tablet 6d67fef331994ca48f1dfce1e3b039f4 (DEFAULT_TABLE table=default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.259045 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67fef331994ca48f1dfce1e3b039f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.261179 620 tablet_service.cc:1511] Processing CreateTablet for tablet 6d67fef331994ca48f1dfce1e3b039f4 (DEFAULT_TABLE table=default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.262300 620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67fef331994ca48f1dfce1e3b039f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.262912 1071 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:06.263293 1071 ts_tablet_manager.cc:1403] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.056s user 0.018s sys 0.001s
I20260810 07:53:06.265656 1071 raft_consensus.cc:359] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.266287 1071 raft_consensus.cc:385] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.266537 1071 raft_consensus.cc:740] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.267367 1071 consensus_queue.cc:260] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.268927 631 tablet_service.cc:1511] Processing CreateTablet for tablet b2fb91a223154dde8ecc2bebf2c0fff9 (DEFAULT_TABLE table=default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.270025 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2fb91a223154dde8ecc2bebf2c0fff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.270255 1071 ts_tablet_manager.cc:1434] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260810 07:53:06.272142 860 tablet_service.cc:1511] Processing CreateTablet for tablet b2fb91a223154dde8ecc2bebf2c0fff9 (DEFAULT_TABLE table=default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.273140 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2fb91a223154dde8ecc2bebf2c0fff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.275100 1071 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:06.277792 376 tablet_service.cc:1511] Processing CreateTablet for tablet 6d67fef331994ca48f1dfce1e3b039f4 (DEFAULT_TABLE table=default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.278734 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67fef331994ca48f1dfce1e3b039f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.278852 856 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7084712ff04ac48a54171534e04d69 (DEFAULT_TABLE table=default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.279938 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7084712ff04ac48a54171534e04d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.281350 1071 tablet_bootstrap.cc:654] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.295611 382 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7084712ff04ac48a54171534e04d69 (DEFAULT_TABLE table=default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.296710 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7084712ff04ac48a54171534e04d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.298648 849 tablet_service.cc:1511] Processing CreateTablet for tablet b705e1630f124370af386436f325004b (DEFAULT_TABLE table=default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.299736 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705e1630f124370af386436f325004b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.304885 1600 consensus_queue.cc:1048] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:06.304991 864 tablet_service.cc:1511] Processing CreateTablet for tablet c47792551bb243b7b3f6de904c17c47b (DEFAULT_TABLE table=default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.305964 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47792551bb243b7b3f6de904c17c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.310783 635 tablet_service.cc:1511] Processing CreateTablet for tablet c47792551bb243b7b3f6de904c17c47b (DEFAULT_TABLE table=default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.311797 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47792551bb243b7b3f6de904c17c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.313769 855 tablet_service.cc:1511] Processing CreateTablet for tablet 93f62494bb034cd0b4f34f61652569c6 (DEFAULT_TABLE table=default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.314821 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f62494bb034cd0b4f34f61652569c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.310767 619 tablet_service.cc:1511] Processing CreateTablet for tablet b705e1630f124370af386436f325004b (DEFAULT_TABLE table=default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.317775 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705e1630f124370af386436f325004b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.320988 623 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7084712ff04ac48a54171534e04d69 (DEFAULT_TABLE table=default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.322252 623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7084712ff04ac48a54171534e04d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.330047 375 tablet_service.cc:1511] Processing CreateTablet for tablet b705e1630f124370af386436f325004b (DEFAULT_TABLE table=default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.331081 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705e1630f124370af386436f325004b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.347491 852 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7 (DEFAULT_TABLE table=default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.349171 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:06.349061 32307 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 132450us
I20260810 07:53:06.357882 385 tablet_service.cc:1511] Processing CreateTablet for tablet c47792551bb243b7b3f6de904c17c47b (DEFAULT_TABLE table=default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.359170 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47792551bb243b7b3f6de904c17c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.362360 1047 tablet_bootstrap.cc:654] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.367501 624 tablet_service.cc:1511] Processing CreateTablet for tablet 93f62494bb034cd0b4f34f61652569c6 (DEFAULT_TABLE table=default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.368634 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f62494bb034cd0b4f34f61652569c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.374233 622 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7 (DEFAULT_TABLE table=default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.375864 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.393601 1600 consensus_queue.cc:1048] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:06.396526 367 tablet_service.cc:1511] Processing CreateTablet for tablet b2fb91a223154dde8ecc2bebf2c0fff9 (DEFAULT_TABLE table=default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.398185 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2fb91a223154dde8ecc2bebf2c0fff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.400964 849 tablet_service.cc:1511] Processing CreateTablet for tablet aed14fa26b9a46a4b3e9a43bac05a795 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.402200 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed14fa26b9a46a4b3e9a43bac05a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.402810 380 tablet_service.cc:1511] Processing CreateTablet for tablet 93f62494bb034cd0b4f34f61652569c6 (DEFAULT_TABLE table=default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.403867 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f62494bb034cd0b4f34f61652569c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.414132 855 tablet_service.cc:1511] Processing CreateTablet for tablet f65480f06a3c4b2886aa2485f81e16f2 (DEFAULT_TABLE table=default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.415450 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65480f06a3c4b2886aa2485f81e16f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.420570 864 tablet_service.cc:1511] Processing CreateTablet for tablet 140522b9e6154f7daca10fdf0c48e6e2 (DEFAULT_TABLE table=default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.421551 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140522b9e6154f7daca10fdf0c48e6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.423254 385 tablet_service.cc:1511] Processing CreateTablet for tablet aed14fa26b9a46a4b3e9a43bac05a795 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.424224 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed14fa26b9a46a4b3e9a43bac05a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.433310 380 tablet_service.cc:1511] Processing CreateTablet for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7 (DEFAULT_TABLE table=default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.434412 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.438139 382 tablet_service.cc:1511] Processing CreateTablet for tablet f65480f06a3c4b2886aa2485f81e16f2 (DEFAULT_TABLE table=default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.439109 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65480f06a3c4b2886aa2485f81e16f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.446225 385 tablet_service.cc:1511] Processing CreateTablet for tablet 140522b9e6154f7daca10fdf0c48e6e2 (DEFAULT_TABLE table=default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.454033 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140522b9e6154f7daca10fdf0c48e6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.458897 633 tablet_service.cc:1511] Processing CreateTablet for tablet 140522b9e6154f7daca10fdf0c48e6e2 (DEFAULT_TABLE table=default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.459335 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:06.459851 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:06.460014 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140522b9e6154f7daca10fdf0c48e6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.467413 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:06.467417 635 tablet_service.cc:1511] Processing CreateTablet for tablet f65480f06a3c4b2886aa2485f81e16f2 (DEFAULT_TABLE table=default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.492071 1064 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:06.492558 1064 ts_tablet_manager.cc:1403] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.490s user 0.010s sys 0.001s
I20260810 07:53:06.499181 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:06.499418 1071 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:06.502179 1071 ts_tablet_manager.cc:1403] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.231s user 0.031s sys 0.025s
I20260810 07:53:06.515618 2285 raft_consensus.cc:493] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.516119 2285 raft_consensus.cc:515] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.517184 1071 raft_consensus.cc:359] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.518079 866 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:06.519088 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:06.519340 2285 leader_election.cc:290] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:06.520023 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d5ab03facf8477eada5d1ac3ec7f5ef already in progress: creating tablet
I20260810 07:53:06.522539 1064 raft_consensus.cc:359] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.456653 627 tablet_service.cc:1511] Processing CreateTablet for tablet aed14fa26b9a46a4b3e9a43bac05a795 (DEFAULT_TABLE table=default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:06.524451 1064 raft_consensus.cc:385] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:53:06.526099 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d5ab03facf8477eada5d1ac3ec7f5ef already in progress: creating tablet
W20260810 07:53:06.522842 308 consensus_peers.cc:597] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:06.529359 1071 raft_consensus.cc:385] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.529699 1071 raft_consensus.cc:740] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.530426 1071 consensus_queue.cc:260] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.532709 1071 ts_tablet_manager.cc:1434] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.030s user 0.002s sys 0.001s
I20260810 07:53:06.541765 1071 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:06.545117 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed14fa26b9a46a4b3e9a43bac05a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:06.525292 1064 raft_consensus.cc:740] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.557325 1064 consensus_queue.cc:260] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.558801 1064 ts_tablet_manager.cc:1434] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.066s user 0.003s sys 0.001s
I20260810 07:53:06.570905 1064 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:06.571008 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d5ab03facf8477eada5d1ac3ec7f5ef already in progress: creating tablet
I20260810 07:53:06.568688 1071 tablet_bootstrap.cc:654] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:06.577559 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 consensus state has no leader
I20260810 07:53:06.581873 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65480f06a3c4b2886aa2485f81e16f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:06.583326 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 46673.8us
I20260810 07:53:06.598219 2103 raft_consensus.cc:493] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.598794 2103 raft_consensus.cc:515] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:06.602061 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 84820.2us
I20260810 07:53:06.604487 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:06.605396 893 raft_consensus.cc:2468] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:06.605792 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:06.606515 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:06.607209 309 leader_election.cc:304] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:06.608369 2229 raft_consensus.cc:2804] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:06.608870 2229 raft_consensus.cc:493] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:06.609333 2229 raft_consensus.cc:3060] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:06.613282 792 leader_election.cc:343] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:06.609325 308 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.615746 2235 raft_consensus.cc:493] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.616209 2235 raft_consensus.cc:515] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.618515 2235 leader_election.cc:290] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:06.621174 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:06.622208 893 raft_consensus.cc:2468] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:06.623840 309 leader_election.cc:304] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:06.624747 2235 raft_consensus.cc:2804] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:06.625221 2235 raft_consensus.cc:493] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:06.625689 2235 raft_consensus.cc:3060] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.626669 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:06.630062 2103 leader_election.cc:290] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:06.634521 792 leader_election.cc:343] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.634995 792 leader_election.cc:304] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:06.641114 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:06.643775 308 leader_election.cc:343] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.624665 1064 tablet_bootstrap.cc:654] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:06.646243 308 consensus_peers.cc:597] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:06.650924 2285 raft_consensus.cc:2749] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:06.653379 2229 raft_consensus.cc:515] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.661310 1047 tablet_bootstrap.cc:492] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:06.661825 1047 ts_tablet_manager.cc:1403] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.774s user 0.016s sys 0.010s
I20260810 07:53:06.664539 1047 raft_consensus.cc:359] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.666765 1047 raft_consensus.cc:385] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.667055 1047 raft_consensus.cc:740] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.671705 2220 consensus_queue.cc:1048] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:06.673002 1047 consensus_queue.cc:260] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:06.678128 309 consensus_peers.cc:597] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:06.685365 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:06.686923 308 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:06.674264 32455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 220356us
I20260810 07:53:06.691785 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:06.692480 892 raft_consensus.cc:3060] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.693323 1071 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:06.693823 1071 ts_tablet_manager.cc:1403] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.152s user 0.008s sys 0.008s
I20260810 07:53:06.694692 2235 raft_consensus.cc:515] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.698844 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:06.704701 1600 consensus_queue.cc:1048] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:06.706079 1047 ts_tablet_manager.cc:1434] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260810 07:53:06.693434 2229 leader_election.cc:290] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:06.711887 892 raft_consensus.cc:2468] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:06.719632 2333 consensus_queue.cc:1048] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:06.732030 1047 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:06.732396 1064 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:06.732872 1064 ts_tablet_manager.cc:1403] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.173s user 0.010s sys 0.005s
I20260810 07:53:06.735985 1064 raft_consensus.cc:359] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:06.702975 308 leader_election.cc:343] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:06.714686 309 leader_election.cc:304] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:06.750468 2229 raft_consensus.cc:2804] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:06.751278 2229 raft_consensus.cc:697] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:06.752172 2229 consensus_queue.cc:237] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.758061 2235 leader_election.cc:290] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:06.696672 1071 raft_consensus.cc:359] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.759268 1071 raft_consensus.cc:385] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.759579 1071 raft_consensus.cc:740] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.762622 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:06.763435 892 raft_consensus.cc:3060] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.760293 1071 consensus_queue.cc:260] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.777680 1071 ts_tablet_manager.cc:1434] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.084s user 0.004s sys 0.002s
I20260810 07:53:06.782339 2285 consensus_queue.cc:1048] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:06.736605 1064 raft_consensus.cc:385] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.784976 1064 raft_consensus.cc:740] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.785642 1064 consensus_queue.cc:260] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.787886 1064 ts_tablet_manager.cc:1434] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.055s user 0.004s sys 0.000s
I20260810 07:53:06.796561 1071 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:06.809468 1047 tablet_bootstrap.cc:654] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.814786 1071 tablet_bootstrap.cc:654] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.826411 2103 consensus_queue.cc:1048] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:06.829378 1047 tablet_bootstrap.cc:492] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:06.829823 1047 ts_tablet_manager.cc:1403] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.098s user 0.006s sys 0.006s
I20260810 07:53:06.846987 1047 raft_consensus.cc:359] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.847869 1047 raft_consensus.cc:385] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.848223 1047 raft_consensus.cc:740] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.860982 1047 consensus_queue.cc:260] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.866348 1047 ts_tablet_manager.cc:1434] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.036s user 0.006s sys 0.002s
I20260810 07:53:06.854995 1064 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:06.873610 1071 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:06.874270 1071 ts_tablet_manager.cc:1403] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.078s user 0.006s sys 0.007s
I20260810 07:53:06.877951 892 raft_consensus.cc:2468] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:06.882506 1071 raft_consensus.cc:359] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.883226 1071 raft_consensus.cc:385] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.883495 1071 raft_consensus.cc:740] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.884145 1047 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:06.889964 1071 consensus_queue.cc:260] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:06.892108 1071 ts_tablet_manager.cc:1434] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260810 07:53:06.893079 1071 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:06.893618 309 leader_election.cc:304] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:06.894824 2229 raft_consensus.cc:2804] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:06.895365 2229 raft_consensus.cc:697] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:06.896739 2229 consensus_queue.cc:237] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.900058 1600 raft_consensus.cc:493] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:06.900528 1600 raft_consensus.cc:515] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.902499 1600 leader_election.cc:290] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:06.904070 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:06.905623 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8718f998623e4f1c8b7b8be815e184a3" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:06.906569 892 raft_consensus.cc:2393] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:06.907987 574 leader_election.cc:304] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:06.908783 1600 raft_consensus.cc:3060] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:06.917742 1600 raft_consensus.cc:2749] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:06.929626 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260810 07:53:06.930866 1047 tablet_bootstrap.cc:654] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.947844 1064 tablet_bootstrap.cc:654] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.966014 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:06.966624 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:06.968340 1071 tablet_bootstrap.cc:654] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:06.978585 1047 tablet_bootstrap.cc:492] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:06.979277 1047 ts_tablet_manager.cc:1403] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.112s user 0.011s sys 0.003s
I20260810 07:53:06.981889 1047 raft_consensus.cc:359] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:06.982347 1064 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:06.982753 1064 ts_tablet_manager.cc:1403] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.128s user 0.009s sys 0.003s
I20260810 07:53:06.985548 1047 raft_consensus.cc:385] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:06.985939 1047 raft_consensus.cc:740] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:06.987444 1047 consensus_queue.cc:260] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.001535 2229 raft_consensus.cc:493] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.002321 2229 raft_consensus.cc:515] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.005064 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:07.005440 1047 ts_tablet_manager.cc:1434] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.026s user 0.005s sys 0.001s
I20260810 07:53:07.005697 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:07.006415 1047 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:07.006620 308 leader_election.cc:343] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.006512 877 raft_consensus.cc:2468] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:07.007650 2229 leader_election.cc:290] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.009476 309 leader_election.cc:304] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:07.014653 1071 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:07.015091 1071 ts_tablet_manager.cc:1403] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.122s user 0.009s sys 0.000s
I20260810 07:53:07.017781 1071 raft_consensus.cc:359] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.018229 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:07.018446 1071 raft_consensus.cc:385] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.018721 1071 raft_consensus.cc:740] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.019348 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:07.019937 1064 raft_consensus.cc:359] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.020750 1064 raft_consensus.cc:385] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.021037 1064 raft_consensus.cc:740] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.021099 32467 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:07.022411 2229 raft_consensus.cc:2804] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:07.022804 2229 raft_consensus.cc:493] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:07.023180 2229 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:07.025295 2350 raft_consensus.cc:493] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.025414 1064 consensus_queue.cc:260] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.025759 2350 raft_consensus.cc:515] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.027741 2350 leader_election.cc:290] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:07.034365 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:07.035560 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:07.036849 792 leader_election.cc:343] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.037010 1071 consensus_queue.cc:260] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:07.037523 792 leader_election.cc:343] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.037860 792 leader_election.cc:304] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:07.038643 1064 ts_tablet_manager.cc:1434] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.056s user 0.007s sys 0.001s
I20260810 07:53:07.038975 2350 raft_consensus.cc:2749] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:07.052039 1071 ts_tablet_manager.cc:1434] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.037s user 0.006s sys 0.001s
I20260810 07:53:07.051810 2229 raft_consensus.cc:515] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.052634 627 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (DEFAULT_TABLE table=default.Table_29e9be4c452b438e82564903ce411e53 [id=81a9ff900b9141c08d9d0d16e795a661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.053793 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4339a26ad540f4b385f69fd0dcdc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.056968 1064 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:07.057548 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:07.058959 308 leader_election.cc:343] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.062667 367 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (DEFAULT_TABLE table=default.Table_29e9be4c452b438e82564903ce411e53 [id=81a9ff900b9141c08d9d0d16e795a661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.063865 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4339a26ad540f4b385f69fd0dcdc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.068434 2235 consensus_queue.cc:1048] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:07.076994 1047 tablet_bootstrap.cc:654] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.083056 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:07.083832 892 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:07.085907 1071 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:07.103571 635 tablet_service.cc:1511] Processing CreateTablet for tablet b6cb01ce3d97454d9d1e7735e51a76c4 (DEFAULT_TABLE table=default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.104920 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cb01ce3d97454d9d1e7735e51a76c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.104856 859 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (DEFAULT_TABLE table=default.Table_29e9be4c452b438e82564903ce411e53 [id=81a9ff900b9141c08d9d0d16e795a661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.110030 2229 leader_election.cc:290] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.114714 380 tablet_service.cc:1511] Processing CreateTablet for tablet b6cb01ce3d97454d9d1e7735e51a76c4 (DEFAULT_TABLE table=default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.115895 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cb01ce3d97454d9d1e7735e51a76c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.116039 892 raft_consensus.cc:2468] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:07.116273 32465 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:07.118849 32465 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:07.121349 309 leader_election.cc:304] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:07.121613 633 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 (DEFAULT_TABLE table=default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.122623 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1ddea2d9f3462092fcbaee7774cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.117982 855 tablet_service.cc:1511] Processing CreateTablet for tablet b6cb01ce3d97454d9d1e7735e51a76c4 (DEFAULT_TABLE table=default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.124845 2235 raft_consensus.cc:2804] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:07.125500 2235 raft_consensus.cc:697] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:07.126754 2235 consensus_queue.cc:237] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.120035 1064 tablet_bootstrap.cc:654] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.129401 382 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 (DEFAULT_TABLE table=default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.130712 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1ddea2d9f3462092fcbaee7774cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.142575 2375 consensus_queue.cc:1048] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:07.121754 32465 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]). Expected version 2 got 0
I20260810 07:53:07.125128 1071 tablet_bootstrap.cc:654] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.159231 849 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 (DEFAULT_TABLE table=default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.187258 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4339a26ad540f4b385f69fd0dcdc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.188012 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1ddea2d9f3462092fcbaee7774cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.192270 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cb01ce3d97454d9d1e7735e51a76c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.209075 385 tablet_service.cc:1511] Processing CreateTablet for tablet e7529f4ac9fb457ba1b3c3bc90eb275d (DEFAULT_TABLE table=default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.211772 1071 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:07.213210 864 tablet_service.cc:1511] Processing CreateTablet for tablet e7529f4ac9fb457ba1b3c3bc90eb275d (DEFAULT_TABLE table=default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.214442 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7529f4ac9fb457ba1b3c3bc90eb275d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.216104 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:07.212286 1071 ts_tablet_manager.cc:1403] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.127s user 0.013s sys 0.002s
I20260810 07:53:07.220921 2229 raft_consensus.cc:493] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.218632 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7529f4ac9fb457ba1b3c3bc90eb275d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.225054 1071 raft_consensus.cc:359] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.225716 1071 raft_consensus.cc:385] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.225976 1071 raft_consensus.cc:740] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.226574 1071 consensus_queue.cc:260] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.228492 1071 ts_tablet_manager.cc:1434] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260810 07:53:07.233024 2229 raft_consensus.cc:515] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.234658 619 tablet_service.cc:1511] Processing CreateTablet for tablet e7529f4ac9fb457ba1b3c3bc90eb275d (DEFAULT_TABLE table=default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.235917 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7529f4ac9fb457ba1b3c3bc90eb275d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.236231 2229 leader_election.cc:290] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:07.213959 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 56759.2us
I20260810 07:53:07.238356 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:07.239359 877 raft_consensus.cc:2468] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:07.241550 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:07.241724 309 leader_election.cc:304] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:07.249455 1071 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:07.253887 1064 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:07.254384 1064 ts_tablet_manager.cc:1403] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.215s user 0.008s sys 0.007s
I20260810 07:53:07.258517 1064 raft_consensus.cc:359] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.254827 2229 raft_consensus.cc:2804] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:07.259169 1064 raft_consensus.cc:385] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.259449 1064 raft_consensus.cc:740] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.259841 2229 raft_consensus.cc:493] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:07.265159 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:07.266572 32464 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:07.273016 380 tablet_service.cc:1511] Processing CreateTablet for tablet 2905f0597c07439989a00a213fc2194f (DEFAULT_TABLE table=default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.276080 1047 tablet_bootstrap.cc:492] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:07.276521 1047 ts_tablet_manager.cc:1403] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.270s user 0.005s sys 0.007s
I20260810 07:53:07.275801 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2905f0597c07439989a00a213fc2194f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.277819 856 tablet_service.cc:1511] Processing CreateTablet for tablet 91c2e1bcd13a4c22b2141949b2d73188 (DEFAULT_TABLE table=default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.279021 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c2e1bcd13a4c22b2141949b2d73188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.279731 1047 raft_consensus.cc:359] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.280360 1047 raft_consensus.cc:385] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.281553 376 tablet_service.cc:1511] Processing CreateTablet for tablet 91c2e1bcd13a4c22b2141949b2d73188 (DEFAULT_TABLE table=default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:07.281973 308 leader_election.cc:343] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.282495 376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c2e1bcd13a4c22b2141949b2d73188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.290643 1071 tablet_bootstrap.cc:654] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.293812 2382 raft_consensus.cc:493] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:07.294473 2382 raft_consensus.cc:515] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.294760 631 tablet_service.cc:1511] Processing CreateTablet for tablet 2905f0597c07439989a00a213fc2194f (DEFAULT_TABLE table=default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.296234 631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2905f0597c07439989a00a213fc2194f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.260167 1064 consensus_queue.cc:260] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.267650 32464 catalog_manager.cc:5719] T f8415b6376a740388487f7ae4aebea52 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:07.270112 2229 raft_consensus.cc:3060] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:07.313949 624 tablet_service.cc:1511] Processing CreateTablet for tablet 91c2e1bcd13a4c22b2141949b2d73188 (DEFAULT_TABLE table=default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.315088 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c2e1bcd13a4c22b2141949b2d73188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.329447 2220 raft_consensus.cc:493] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.330067 2220 raft_consensus.cc:515] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.333000 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:07.336095 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:07.336676 2220 leader_election.cc:290] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.338665 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:07.339380 32468 catalog_manager.cc:5719] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:07.341753 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:07.342473 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:07.344125 32460 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 268) took 3501 ms. Trace:
I20260810 07:53:07.347460 375 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.348454 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8f390b0db64e1d96c3d8686274cbbf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:07.361843 308 leader_election.cc:343] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:07.367719 635 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.368144 382 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25b4b974034eea9f1256fb4e2a503c (DEFAULT_TABLE table=default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.369043 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8f390b0db64e1d96c3d8686274cbbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.369138 382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25b4b974034eea9f1256fb4e2a503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.373545 627 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25b4b974034eea9f1256fb4e2a503c (DEFAULT_TABLE table=default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.375207 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25b4b974034eea9f1256fb4e2a503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.379786 2382 leader_election.cc:290] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.344431 32460 rpcz_store.cc:276] 0810 07:53:03.842469 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.842598 (+ 129us) service_pool.cc:224] Handling call
0810 07:53:03.844834 (+ 2236us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c846 00007f30cf429634 00007f30cf4295c2 00007f30cf4294f1 00007f30cf42946c 00007f30cf428722 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:03.934479 (+ 89645us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:03.936006 (+ 1527us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:03.936578 (+ 572us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:03.936901 (+ 323us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:04.564642 (+627741us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:07.344065 (+2779423us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.456689 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.463665 (+ 6976us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.463692 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:53:04.463704 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:04.464020 (+ 316us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.464061 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.464084 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:04.464237 (+ 153us) tablet.cc:701] Row locks acquired
0810 07:53:04.464251 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:04.464371 (+ 120us) write_op.cc:270] Start()
0810 07:53:04.464458 (+ 87us) write_op.cc:276] Timestamp: P: 1786348384464332 usec, L: 0
0810 07:53:04.464477 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.471296 (+ 6819us) log.cc:844] Serialized 405 byte log entry
0810 07:53:04.548442 (+ 77146us) op_driver.cc:464] REPLICATION: finished
0810 07:53:04.550586 (+ 2144us) write_op.cc:301] APPLY: starting
0810 07:53:04.550651 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:04.550742 (+ 91us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:04.550948 (+ 206us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:04.551182 (+ 234us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:04.551318 (+ 136us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:04.551363 (+ 45us) write_op.cc:312] APPLY: finished
0810 07:53:04.551559 (+ 196us) log.cc:844] Serialized 23 byte log entry
0810 07:53:04.562888 (+ 11329us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:04.563178 (+ 290us) write_op.cc:454] Released schema lock
0810 07:53:04.563418 (+ 240us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:04.569339 (+ 5921us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:04.569368 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":3302144,"child_traces":[["op",{"apply.queue_time_us":1965,"apply.run_cpu_time_us":1719,"apply.run_wall_time_us":18720,"num_ops":1,"prepare.queue_time_us":518626,"prepare.run_cpu_time_us":2266,"prepare.run_wall_time_us":16138,"raft.queue_time_us":6081,"raft.run_cpu_time_us":1044,"raft.run_wall_time_us":1044,"replication_time_us":83855}]]}
W20260810 07:53:07.391682 32460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 30) took 1027 ms (client timeout 1000 ms). Trace:
W20260810 07:53:07.393260 32460 rpcz_store.cc:269] 0810 07:53:06.364771 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.391145 (+1026374us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:07.391591 (+ 446us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:07.394474 1071 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:07.394984 1071 ts_tablet_manager.cc:1403] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.146s user 0.024s sys 0.002s
I20260810 07:53:07.398729 1071 raft_consensus.cc:359] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.399516 1071 raft_consensus.cc:385] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.399792 1071 raft_consensus.cc:740] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.405798 32469 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 285) took 1578 ms. Trace:
I20260810 07:53:07.415609 1064 ts_tablet_manager.cc:1434] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.161s user 0.009s sys 0.000s
I20260810 07:53:07.400476 1071 consensus_queue.cc:260] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.422827 1071 ts_tablet_manager.cc:1434] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260810 07:53:07.423777 1071 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:07.406145 32469 rpcz_store.cc:276] 0810 07:53:05.827356 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:06.107270 (+279914us) service_pool.cc:224] Handling call
0810 07:53:06.108445 (+ 1175us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:06.109232 (+ 787us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:06.109779 (+ 547us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:06.110001 (+ 222us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:06.795472 (+685471us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:07.405736 (+610264us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:06.453564 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:06.453931 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:06.453959 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:06.453977 (+ 18us) tablet.cc:662] Decoding operations
0810 07:53:06.505281 (+ 51304us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:06.505355 (+ 74us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:06.505380 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:06.505703 (+ 323us) tablet.cc:701] Row locks acquired
0810 07:53:06.505718 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:06.505972 (+ 254us) write_op.cc:270] Start()
0810 07:53:06.506164 (+ 192us) write_op.cc:276] Timestamp: P: 1786348386505892 usec, L: 0
0810 07:53:06.506182 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:06.507187 (+ 1005us) log.cc:844] Serialized 405 byte log entry
0810 07:53:06.768556 (+261369us) op_driver.cc:464] REPLICATION: finished
0810 07:53:06.771127 (+ 2571us) write_op.cc:301] APPLY: starting
0810 07:53:06.771195 (+ 68us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:06.771315 (+ 120us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:06.771406 (+ 91us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:06.771632 (+ 226us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:06.771802 (+ 170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:06.771846 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:06.772205 (+ 359us) log.cc:844] Serialized 23 byte log entry
0810 07:53:06.772504 (+ 299us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:06.772624 (+ 120us) write_op.cc:454] Released schema lock
0810 07:53:06.772678 (+ 54us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2435,"apply.run_cpu_time_us":1879,"apply.run_wall_time_us":24359,"num_ops":1,"prepare.queue_time_us":79407,"prepare.run_cpu_time_us":2519,"prepare.run_wall_time_us":54060,"replication_time_us":262267,"thread_start_us":81701,"threads_started":1}]]}
I20260810 07:53:07.424853 2375 raft_consensus.cc:493] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.425412 2375 raft_consensus.cc:515] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.433957 2375 leader_election.cc:290] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.280611 1047 raft_consensus.cc:740] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.439211 1047 consensus_queue.cc:260] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.441265 32470 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1] alter complete (version 1)
I20260810 07:53:07.441598 1047 ts_tablet_manager.cc:1434] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.165s user 0.005s sys 0.000s
I20260810 07:53:07.453282 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8415b6376a740388487f7ae4aebea52" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:07.455065 866 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25b4b974034eea9f1256fb4e2a503c (DEFAULT_TABLE table=default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.457635 860 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.458536 866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25b4b974034eea9f1256fb4e2a503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.459086 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8f390b0db64e1d96c3d8686274cbbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.465080 574 leader_election.cc:304] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:07.465848 622 tablet_service.cc:1511] Processing CreateTablet for tablet 94ced893d6804aa395b73f1f5ed085f9 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.465981 2334 raft_consensus.cc:2749] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:07.468075 380 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.478399 367 tablet_service.cc:1511] Processing CreateTablet for tablet 94ced893d6804aa395b73f1f5ed085f9 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.480137 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94ced893d6804aa395b73f1f5ed085f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.485275 2375 raft_consensus.cc:493] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.485769 2375 raft_consensus.cc:515] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.463229 2350 raft_consensus.cc:493] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.488888 2375 leader_election.cc:290] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:07.488981 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:07.489090 2350 raft_consensus.cc:515] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.489734 1071 tablet_bootstrap.cc:654] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:07.490688 308 leader_election.cc:343] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.492218 676 raft_consensus.cc:3060] T f8415b6376a740388487f7ae4aebea52 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:07.496440 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9be900b701a45728a44bf26350a26ab" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:07.498029 308 leader_election.cc:343] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.501112 2350 leader_election.cc:290] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:07.504191 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:07.505249 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:07.505852 409 raft_consensus.cc:2468] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
W20260810 07:53:07.506187 792 leader_election.cc:343] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.510340 859 tablet_service.cc:1511] Processing CreateTablet for tablet 94ced893d6804aa395b73f1f5ed085f9 (DEFAULT_TABLE table=default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.511390 859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94ced893d6804aa395b73f1f5ed085f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.513000 862 tablet_service.cc:1511] Processing CreateTablet for tablet e0a74b22a6864180956e763d2b6b6de6 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.514401 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0a74b22a6864180956e763d2b6b6de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.522677 1064 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:07.526228 1047 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:07.526399 792 leader_election.cc:304] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:07.528017 2350 raft_consensus.cc:2804] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:07.528353 2350 raft_consensus.cc:493] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:07.528764 2350 raft_consensus.cc:3060] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:07.537077 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.538853 1071 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:07.539358 1071 ts_tablet_manager.cc:1403] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.116s user 0.007s sys 0.007s
I20260810 07:53:07.542924 633 tablet_service.cc:1511] Processing CreateTablet for tablet e0a74b22a6864180956e763d2b6b6de6 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.543110 1071 raft_consensus.cc:359] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.543824 1071 raft_consensus.cc:385] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.544138 1071 raft_consensus.cc:740] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.545392 633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0a74b22a6864180956e763d2b6b6de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.545992 2395 tablet.cc:1724] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:07.547787 385 tablet_service.cc:1511] Processing CreateTablet for tablet e0a74b22a6864180956e763d2b6b6de6 (DEFAULT_TABLE table=default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.570127 864 tablet_service.cc:1511] Processing CreateTablet for tablet 2905f0597c07439989a00a213fc2194f (DEFAULT_TABLE table=default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.557045 1071 consensus_queue.cc:260] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.574837 1071 ts_tablet_manager.cc:1434] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.035s user 0.006s sys 0.002s
I20260810 07:53:07.581445 369 tablet_service.cc:1511] Processing CreateTablet for tablet 8ee78955a68a451f81e01a62a57aa4a4 (DEFAULT_TABLE table=default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.581610 855 tablet_service.cc:1511] Processing CreateTablet for tablet 8ee78955a68a451f81e01a62a57aa4a4 (DEFAULT_TABLE table=default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.583634 371 tablet_service.cc:1511] Processing CreateTablet for tablet bdc0039db22d4812a1412e10175d2237 (DEFAULT_TABLE table=default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.584914 2229 raft_consensus.cc:515] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.590204 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.591267 377 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.599246 2229 leader_election.cc:290] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:07.606351 624 tablet_service.cc:1511] Processing CreateTablet for tablet 8ee78955a68a451f81e01a62a57aa4a4 (DEFAULT_TABLE table=default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.607694 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee78955a68a451f81e01a62a57aa4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.619596 619 tablet_service.cc:1511] Processing CreateTablet for tablet bdc0039db22d4812a1412e10175d2237 (DEFAULT_TABLE table=default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.621206 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc0039db22d4812a1412e10175d2237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.630826 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee78955a68a451f81e01a62a57aa4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.637131 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2905f0597c07439989a00a213fc2194f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.639269 2394 tablet.cc:1724] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:07.644968 32467 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e] alter complete (version 2)
W20260810 07:53:07.651284 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 40) took 3970 ms (client timeout 1000 ms). Trace:
I20260810 07:53:07.671156 2334 raft_consensus.cc:493] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.674295 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0a74b22a6864180956e763d2b6b6de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.674522 1071 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:07.676608 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ee78955a68a451f81e01a62a57aa4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.672539 2334 raft_consensus.cc:515] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.678905 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc0039db22d4812a1412e10175d2237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.689806 1071 tablet_bootstrap.cc:654] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.701627 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 449359c8e9fa4756ba149f3dce6e0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5c3bed1f2a42b1bbf18b497555eb7a) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.701987 377 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.703080 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.709542 1047 tablet_bootstrap.cc:654] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.647127 866 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.655395 856 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.656515 862 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.658306 860 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.659487 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.661609 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.723594 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94ced893d6804aa395b73f1f5ed085f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.726665 1071 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:07.727319 1071 ts_tablet_manager.cc:1403] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.008s
I20260810 07:53:07.734966 862 tablet_service.cc:1511] Processing CreateTablet for tablet bdc0039db22d4812a1412e10175d2237 (DEFAULT_TABLE table=default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.736641 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc0039db22d4812a1412e10175d2237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.762864 1064 tablet_bootstrap.cc:654] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:07.778076 2334 leader_election.cc:290] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:07.793448 377 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:07.793905 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:07.663661 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 26) took 4693 ms (client timeout 1000 ms). Trace:
W20260810 07:53:07.808578 32463 rpcz_store.cc:269] 0810 07:53:02.970092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.085025 (+114933us) service_pool.cc:224] Handling call
0810 07:53:03.137231 (+ 52206us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:07.663571 (+4526340us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:04.046018 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:04.046797 (+ 779us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:04.046834 (+ 37us) write_op.cc:435] Acquired schema lock
0810 07:53:04.046854 (+ 20us) tablet.cc:662] Decoding operations
0810 07:53:04.047226 (+ 372us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:04.047282 (+ 56us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:04.047319 (+ 37us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:04.047543 (+ 224us) tablet.cc:701] Row locks acquired
0810 07:53:04.047564 (+ 21us) write_op.cc:260] PREPARE: finished
0810 07:53:04.047717 (+ 153us) write_op.cc:270] Start()
0810 07:53:04.047836 (+ 119us) write_op.cc:276] Timestamp: P: 1786348384047651 usec, L: 0
0810 07:53:04.047861 (+ 25us) op_driver.cc:348] REPLICATION: starting
0810 07:53:04.048564 (+ 703us) log.cc:844] Serialized 533 byte log entry
0810 07:53:04.218500 (+169936us) op_driver.cc:464] REPLICATION: finished
0810 07:53:04.219323 (+ 823us) write_op.cc:301] APPLY: starting
0810 07:53:04.219400 (+ 77us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:04.219483 (+ 83us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:04.219571 (+ 88us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:04.219765 (+ 194us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:04.220014 (+ 249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:04.220088 (+ 74us) write_op.cc:312] APPLY: finished
0810 07:53:04.220348 (+ 260us) log.cc:844] Serialized 23 byte log entry
0810 07:53:04.220544 (+ 196us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:04.220616 (+ 72us) write_op.cc:454] Released schema lock
0810 07:53:04.220656 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:04.222397 (+ 1741us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:04.222436 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":120939,"reported_tablets":26,"spinlock_wait_cycles":110313728,"child_traces":[["op",{"apply.queue_time_us":656,"apply.run_cpu_time_us":1684,"apply.run_wall_time_us":3001,"num_ops":1,"prepare.queue_time_us":389905,"prepare.run_cpu_time_us":2673,"prepare.run_wall_time_us":4289,"replication_time_us":170524}]]}
I20260810 07:53:07.821885 1047 tablet_bootstrap.cc:492] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:07.822319 1047 ts_tablet_manager.cc:1403] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.296s user 0.007s sys 0.008s
I20260810 07:53:07.830240 635 tablet_service.cc:1558] Processing DeleteTablet for tablet d7626d30f4574658bb226f2e915ee0de with delete_type TABLET_DATA_DELETED (replaced by tablet c47792551bb243b7b3f6de904c17c47b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.838860 2350 raft_consensus.cc:515] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.860761 2350 leader_election.cc:290] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:07.862061 627 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.862890 2417 tablet_replica.cc:333] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:07.863796 2417 raft_consensus.cc:2243] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:07.864581 2417 raft_consensus.cc:2272] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:07.864670 1047 raft_consensus.cc:359] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.865340 1047 raft_consensus.cc:385] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.865585 1047 raft_consensus.cc:740] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.867035 2417 ts_tablet_manager.cc:1916] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:07.880505 2403 raft_consensus.cc:493] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.881107 2403 raft_consensus.cc:515] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:07.897289 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:07.898360 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:07.899682 792 leader_election.cc:343] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.900825 2403 leader_election.cc:290] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:07.900410 792 leader_election.cc:343] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:07.908360 1047 consensus_queue.cc:260] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.913511 1071 raft_consensus.cc:359] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.914256 1071 raft_consensus.cc:385] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:07.914538 1071 raft_consensus.cc:740] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:07.919795 630 tablet_service.cc:1558] Processing DeleteTablet for tablet a89cdc44a8c64ccd83fd30855798f87b with delete_type TABLET_DATA_DELETED (replaced by tablet aed14fa26b9a46a4b3e9a43bac05a795) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.920135 620 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c171d3358e44ae2962ab3c4c377b0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:55 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.919821 631 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.920972 623 tablet_service.cc:1558] Processing DeleteTablet for tablet c1faed468bfa4d1ba5d92c0b9b194781 with delete_type TABLET_DATA_DELETED (replaced by tablet cec929b5f0e14f6586ed50137ecd741b) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.921620 645 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.924528 635 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:07.926646 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e4b3ca46c9c44559761e25bcde3e927 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:07.739136 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 5e4b3ca46c9c44559761e25bcde3e927 because the tablet was not found. No further retry: Not found: Tablet not found: 5e4b3ca46c9c44559761e25bcde3e927
W20260810 07:53:07.929098 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 189982us
W20260810 07:53:07.931416 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet ed74e441397344e09e924fb724ad55d6 (table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ed74e441397344e09e924fb724ad55d6 consensus state has no leader
W20260810 07:53:07.931852 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet dd22d1029ff249a0a9e05862293261ff because the tablet was not found. No further retry: Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:07.934259 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.935220 866 tablet_service.cc:1558] Processing DeleteTablet for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 04aff252a46447f49c59345035374580) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.935652 852 tablet_service.cc:1511] Processing CreateTablet for tablet f6cd26a04d3c417d87091f05bc5cb5cd (DEFAULT_TABLE table=default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.935917 851 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:07.937001 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cd26a04d3c417d87091f05bc5cb5cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.937281 860 tablet_service.cc:1511] Processing CreateTablet for tablet a6077111ffb2411892b62d806ea3c85b (DEFAULT_TABLE table=default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.938328 860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6077111ffb2411892b62d806ea3c85b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.938249 627 tablet_service.cc:1511] Processing CreateTablet for tablet a6077111ffb2411892b62d806ea3c85b (DEFAULT_TABLE table=default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.939798 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6077111ffb2411892b62d806ea3c85b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:07.941675 856 tablet_service.cc:1511] Processing CreateTablet for tablet 04aff252a46447f49c59345035374580 (DEFAULT_TABLE table=default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:07.942844 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04aff252a46447f49c59345035374580. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:07.932420 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet d7626d30f4574658bb226f2e915ee0de because the tablet was not found. No further retry: Not found: Tablet not found: d7626d30f4574658bb226f2e915ee0de
W20260810 07:53:07.946041 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 37b48c88571040f6be8d7cbd388feb58 because the tablet was not found. No further retry: Not found: Tablet not found: 37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:07.947470 2412 tablet_replica.cc:333] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:07.950590 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:07.951339 896 raft_consensus.cc:2393] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:07.854586 2382 raft_consensus.cc:493] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:53:07.711236 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 41) took 2924 ms (client timeout 1000 ms). Trace:
I20260810 07:53:07.952845 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9be900b701a45728a44bf26350a26ab" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:07.953375 896 raft_consensus.cc:2468] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:07.951330 1071 consensus_queue.cc:260] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:07.952934 32462 rpcz_store.cc:269] 0810 07:53:04.786480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.107876 (+321396us) service_pool.cc:224] Handling call
0810 07:53:07.711139 (+2603263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:06.449063 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:06.449459 (+ 396us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:06.449484 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:53:06.449496 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:06.449767 (+ 271us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:06.449803 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:06.449824 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:06.449964 (+ 140us) tablet.cc:701] Row locks acquired
0810 07:53:06.449977 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:06.450084 (+ 107us) write_op.cc:270] Start()
0810 07:53:06.450163 (+ 79us) write_op.cc:276] Timestamp: P: 1786348386450002 usec, L: 0
0810 07:53:06.450177 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:06.450887 (+ 710us) log.cc:844] Serialized 533 byte log entry
0810 07:53:06.469939 (+ 19052us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5800090bd0. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cf95581c 00007f30cf95c133 00007f30cf95c0b9 00007f30cf95c049 00007f30cf95c011 00007f30cf95b1dd 00000000005bd8c4 00000000005bd6f9 00007f30c8895db5 00007f30c8898121
0810 07:53:06.768279 (+298340us) op_driver.cc:464] REPLICATION: finished
0810 07:53:06.769188 (+ 909us) write_op.cc:301] APPLY: starting
0810 07:53:06.769270 (+ 82us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:06.769366 (+ 96us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:06.769387 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:06.769561 (+ 174us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:06.769725 (+ 164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:06.769845 (+ 120us) write_op.cc:312] APPLY: finished
0810 07:53:06.770147 (+ 302us) log.cc:844] Serialized 23 byte log entry
0810 07:53:06.770397 (+ 250us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:06.770494 (+ 97us) write_op.cc:454] Released schema lock
0810 07:53:06.770534 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:06.770823 (+ 289us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:06.770853 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":627,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":729,"apply.run_cpu_time_us":1586,"apply.run_wall_time_us":1586,"num_ops":1,"prepare.queue_time_us":239907,"prepare.run_cpu_time_us":1934,"prepare.run_wall_time_us":4216,"raft.queue_time_us":81,"raft.run_cpu_time_us":833,"raft.run_wall_time_us":16899,"replication_time_us":317961,"spinlock_wait_cycles":37507840}]]}
W20260810 07:53:07.821807 32461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 28) took 4009 ms (client timeout 1000 ms). Trace:
W20260810 07:53:07.839195 32467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 27) took 2807 ms (client timeout 1000 ms). Trace:
I20260810 07:53:07.970366 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:07.971045 896 raft_consensus.cc:3060] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:07.968890 32467 rpcz_store.cc:269] 0810 07:53:05.032044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.161489 (+129445us) service_pool.cc:224] Handling call
0810 07:53:05.298131 (+136642us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:07.644540 (+2346409us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.644565 (+ 25us) write_op.cc:454] Released schema lock
0810 07:53:07.839117 (+194552us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:07.613395 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:07.613813 (+ 418us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:07.613839 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:07.613850 (+ 11us) tablet.cc:662] Decoding operations
0810 07:53:07.614201 (+ 351us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:07.614244 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:07.614266 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:07.614427 (+ 161us) tablet.cc:701] Row locks acquired
0810 07:53:07.614439 (+ 12us) write_op.cc:260] PREPARE: finished
0810 07:53:07.614557 (+ 118us) write_op.cc:270] Start()
0810 07:53:07.614647 (+ 90us) write_op.cc:276] Timestamp: P: 1786348387614514 usec, L: 0
0810 07:53:07.614665 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.615210 (+ 545us) log.cc:844] Serialized 356 byte log entry
0810 07:53:07.625508 (+ 10298us) op_driver.cc:464] REPLICATION: finished
0810 07:53:07.625942 (+ 434us) write_op.cc:301] APPLY: starting
0810 07:53:07.626031 (+ 89us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:07.626167 (+ 136us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:07.626293 (+ 126us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:07.626616 (+ 323us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:07.626803 (+ 187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:07.626888 (+ 85us) write_op.cc:312] APPLY: finished
0810 07:53:07.627129 (+ 241us) log.cc:844] Serialized 23 byte log entry
0810 07:53:07.627329 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.627425 (+ 96us) write_op.cc:454] Released schema lock
0810 07:53:07.627471 (+ 46us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":52181,"reported_tablets":27,"spinlock_wait_cycles":251276032,"child_traces":[["op",{"apply.queue_time_us":233,"apply.run_cpu_time_us":1773,"apply.run_wall_time_us":1772,"num_ops":1,"prepare.queue_time_us":26544,"prepare.run_cpu_time_us":2107,"prepare.run_wall_time_us":3362,"raft.queue_time_us":217,"raft.run_cpu_time_us":772,"raft.run_wall_time_us":772,"replication_time_us":10707}]]}
I20260810 07:53:07.977402 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_8d4b32c5738e41c197473d44c3151c37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:07.970815 2375 raft_consensus.cc:493] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.982682 2375 raft_consensus.cc:515] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:07.862768 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 102577us
I20260810 07:53:07.988468 2407 raft_consensus.cc:493] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:07.989156 2407 raft_consensus.cc:515] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:07.992002 2407 leader_election.cc:290] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:08.005004 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:08.005646 414 raft_consensus.cc:2468] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:07.955809 2382 raft_consensus.cc:515] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:08.001636 849 consensus_meta.cc:287] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.749s user 0.000s sys 0.009s
W20260810 07:53:08.022305 2409 log.cc:927] Time spent T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260810 07:53:08.009663 2375 leader_election.cc:290] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:08.033193 2397 tablet_replica.cc:333] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:07.993749 2412 raft_consensus.cc:2243] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:08.038546 2382 leader_election.cc:290] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:08.047338 1071 ts_tablet_manager.cc:1434] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.320s user 0.004s sys 0.012s
I20260810 07:53:08.049314 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:08.061530 377 tablet_service.cc:1511] Processing CreateTablet for tablet 04aff252a46447f49c59345035374580 (DEFAULT_TABLE table=default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:08.062712 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04aff252a46447f49c59345035374580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:08.078099 375 tablet_service.cc:1511] Processing CreateTablet for tablet f6cd26a04d3c417d87091f05bc5cb5cd (DEFAULT_TABLE table=default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:08.079843 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cd26a04d3c417d87091f05bc5cb5cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:08.048285 2412 raft_consensus.cc:2272] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:08.087513 385 tablet_service.cc:1511] Processing CreateTablet for tablet a6077111ffb2411892b62d806ea3c85b (DEFAULT_TABLE table=default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:08.089013 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6077111ffb2411892b62d806ea3c85b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:07.956586 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because the tablet was not found. No further retry: Not found: Tablet not found: 54a3bd8369634e4bbfdf8920454357bc
W20260810 07:53:08.099036 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 142465us
I20260810 07:53:08.097100 635 tablet_service.cc:1511] Processing CreateTablet for tablet f6cd26a04d3c417d87091f05bc5cb5cd (DEFAULT_TABLE table=default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:08.096315 622 tablet_service.cc:1511] Processing CreateTablet for tablet 04aff252a46447f49c59345035374580 (DEFAULT_TABLE table=default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:07.815599 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 29) took 2469 ms (client timeout 1000 ms). Trace:
I20260810 07:53:08.100601 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cd26a04d3c417d87091f05bc5cb5cd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:08.101526 32465 rpcz_store.cc:269] 0810 07:53:05.346116 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.393814 (+ 47698us) service_pool.cc:224] Handling call
0810 07:53:05.463055 (+ 69241us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:07.815527 (+2352472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93603,"reported_tablets":26,"spinlock_wait_cycles":149572480}
I20260810 07:53:08.103896 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:08.105224 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04aff252a46447f49c59345035374580. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:08.106082 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:07.979189 32463 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_9a4d68bb16a74d4786e2176453635dc7" } modify_external_catalogs: true
W20260810 07:53:07.996354 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet d7626d30f4574658bb226f2e915ee0de because the tablet was not found. No further retry: Not found: Tablet not found: d7626d30f4574658bb226f2e915ee0de
W20260810 07:53:08.107252 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 110797us
W20260810 07:53:07.801043 32470 rpcz_store.cc:269] 0810 07:53:03.681281 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.681398 (+ 117us) service_pool.cc:224] Handling call
0810 07:53:07.651202 (+3969804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:05.374482 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:05.374865 (+ 383us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:05.374888 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:05.374901 (+ 13us) tablet.cc:662] Decoding operations
0810 07:53:05.375148 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:05.375181 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:05.375202 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:05.375349 (+ 147us) tablet.cc:701] Row locks acquired
0810 07:53:05.375367 (+ 18us) write_op.cc:260] PREPARE: finished
0810 07:53:05.375478 (+ 111us) write_op.cc:270] Start()
0810 07:53:05.375555 (+ 77us) write_op.cc:276] Timestamp: P: 1786348385375405 usec, L: 0
0810 07:53:05.375569 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:05.376055 (+ 486us) log.cc:844] Serialized 533 byte log entry
0810 07:53:05.429670 (+ 53615us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.432346 (+ 2676us) write_op.cc:301] APPLY: starting
0810 07:53:05.432404 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.432531 (+ 127us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.432633 (+ 102us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.437086 (+ 4453us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.437263 (+ 177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:05.437326 (+ 63us) write_op.cc:312] APPLY: finished
0810 07:53:05.437572 (+ 246us) log.cc:844] Serialized 23 byte log entry
0810 07:53:05.437725 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.437783 (+ 58us) write_op.cc:454] Released schema lock
0810 07:53:05.437824 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:53:07.374238 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:07.374644 (+ 406us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:07.374671 (+ 27us) write_op.cc:435] Acquired schema lock
0810 07:53:07.374683 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:07.374963 (+ 280us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:07.375005 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:07.375028 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:07.375172 (+ 144us) tablet.cc:701] Row locks acquired
0810 07:53:07.375185 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:07.375306 (+ 121us) write_op.cc:270] Start()
0810 07:53:07.375401 (+ 95us) write_op.cc:276] Timestamp: P: 1786348387375230 usec, L: 0
0810 07:53:07.375416 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.421955 (+ 46539us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b5400030b18. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30d07a0964 00007f30cf9b6d19 00007f30cf972a01 00007f30cf9726da 00007f30cf9f6d45 00007f30cf9f71e9 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319
0810 07:53:07.422194 (+ 239us) log.cc:844] Serialized 356 byte log entry
0810 07:53:07.435879 (+ 13685us) op_driver.cc:464] REPLICATION: finished
0810 07:53:07.436264 (+ 385us) write_op.cc:301] APPLY: starting
0810 07:53:07.436350 (+ 86us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:07.436455 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:07.436480 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:07.436769 (+ 289us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:07.436951 (+ 182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:07.437036 (+ 85us) write_op.cc:312] APPLY: finished
0810 07:53:07.437257 (+ 221us) log.cc:844] Serialized 23 byte log entry
0810 07:53:07.437450 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.437521 (+ 71us) write_op.cc:454] Released schema lock
0810 07:53:07.437562 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":42474,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":2546,"apply.run_cpu_time_us":1598,"apply.run_wall_time_us":12575,"num_ops":1,"prepare.queue_time_us":273316,"prepare.run_cpu_time_us":1807,"prepare.run_wall_time_us":1808,"raft.queue_time_us":6026,"raft.run_cpu_time_us":799,"raft.run_wall_time_us":800,"replication_time_us":54022}],["op",{"apply.queue_time_us":183,"apply.run_cpu_time_us":1521,"apply.run_wall_time_us":1521,"num_ops":1,"prepare.queue_time_us":13479,"prepare.run_cpu_time_us":3253,"prepare.run_wall_time_us":52871,"raft.queue_time_us":2381,"raft.run_cpu_time_us":1169,"raft.run_wall_time_us":1169,"replication_time_us":60341,"spinlock_wait_cycles":105151872}]]}
I20260810 07:53:08.112036 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_c21adb23b99c477d9f9ea8bf2fc0558f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:08.114548 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c21adb23b99c477d9f9ea8bf2fc0558f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:08.114647 2417 ts_tablet_manager.cc:1929] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:08.107910 896 raft_consensus.cc:2468] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:08.123623 1064 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:08.124154 1064 ts_tablet_manager.cc:1403] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.626s user 0.015s sys 0.005s
I20260810 07:53:08.127635 1071 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:08.130127 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:08.144228 2417 log.cc:1199] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/54a3bd8369634e4bbfdf8920454357bc
W20260810 07:53:08.144449 2427 log.cc:927] Time spent T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.066s user 0.001s sys 0.000s
I20260810 07:53:08.145605 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:08.147100 2417 ts_tablet_manager.cc:1950] T 54a3bd8369634e4bbfdf8920454357bc P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:08.157181 1600 raft_consensus.cc:493] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:08.157629 1600 raft_consensus.cc:515] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:07.917037 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 43) took 2007 ms (client timeout 1000 ms). Trace:
I20260810 07:53:08.161550 1047 ts_tablet_manager.cc:1434] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.339s user 0.007s sys 0.000s
I20260810 07:53:08.160526 1600 leader_election.cc:290] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:08.161000 32464 rpcz_store.cc:269] 0810 07:53:05.909429 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:05.946716 (+ 37287us) service_pool.cc:224] Handling call
0810 07:53:07.916952 (+1970236us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:07.302026 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:07.302414 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:07.302440 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:07.302451 (+ 11us) tablet.cc:662] Decoding operations
0810 07:53:07.302703 (+ 252us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:07.302744 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:07.302780 (+ 36us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:07.302941 (+ 161us) tablet.cc:701] Row locks acquired
0810 07:53:07.302958 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:53:07.303076 (+ 118us) write_op.cc:270] Start()
0810 07:53:07.303159 (+ 83us) write_op.cc:276] Timestamp: P: 1786348387303008 usec, L: 0
0810 07:53:07.303173 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.303693 (+ 520us) log.cc:844] Serialized 533 byte log entry
0810 07:53:07.316632 (+ 12939us) op_driver.cc:464] REPLICATION: finished
0810 07:53:07.317159 (+ 527us) write_op.cc:301] APPLY: starting
0810 07:53:07.317222 (+ 63us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:07.317313 (+ 91us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:07.317410 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:07.317581 (+ 171us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:07.317730 (+ 149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:07.317771 (+ 41us) write_op.cc:312] APPLY: finished
0810 07:53:07.317945 (+ 174us) log.cc:844] Serialized 23 byte log entry
0810 07:53:07.318097 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.318166 (+ 69us) write_op.cc:454] Released schema lock
0810 07:53:07.318201 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:07.318454 (+ 253us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.318479 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90906,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":359,"apply.run_cpu_time_us":1261,"apply.run_wall_time_us":1260,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":1981,"prepare.run_wall_time_us":1981,"raft.queue_time_us":3422,"raft.run_cpu_time_us":700,"raft.run_wall_time_us":701,"replication_time_us":13316}]]}
I20260810 07:53:08.164642 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 6a97561b5ff5490890e756ca5ff50d38 from {username='slave'} at 127.0.0.1:43806
W20260810 07:53:07.965011 32461 rpcz_store.cc:269] 0810 07:53:03.811923 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:03.812065 (+ 142us) service_pool.cc:224] Handling call
0810 07:53:03.845428 (+ 33363us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:07.821733 (+3976305us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:05.668951 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:05.669730 (+ 779us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:05.669755 (+ 25us) write_op.cc:435] Acquired schema lock
0810 07:53:05.669768 (+ 13us) tablet.cc:662] Decoding operations
0810 07:53:05.670042 (+ 274us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:05.670085 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:05.670110 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:05.670259 (+ 149us) tablet.cc:701] Row locks acquired
0810 07:53:05.670274 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:05.670386 (+ 112us) write_op.cc:270] Start()
0810 07:53:05.670476 (+ 90us) write_op.cc:276] Timestamp: P: 1786348385670330 usec, L: 0
0810 07:53:05.670495 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:05.671000 (+ 505us) log.cc:844] Serialized 533 byte log entry
0810 07:53:05.721070 (+ 50070us) op_driver.cc:464] REPLICATION: finished
0810 07:53:05.741469 (+ 20399us) write_op.cc:301] APPLY: starting
0810 07:53:05.741534 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:05.741622 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:05.741717 (+ 95us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:05.741941 (+ 224us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:05.742066 (+ 125us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:05.742130 (+ 64us) write_op.cc:312] APPLY: finished
0810 07:53:05.742310 (+ 180us) log.cc:844] Serialized 23 byte log entry
0810 07:53:05.742468 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.742530 (+ 62us) write_op.cc:454] Released schema lock
0810 07:53:05.742561 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:05.742819 (+ 258us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:05.742843 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":128370,"reported_tablets":25,"spinlock_wait_cycles":63670656,"child_traces":[["op",{"apply.queue_time_us":20224,"apply.run_cpu_time_us":1317,"apply.run_wall_time_us":1316,"num_ops":1,"prepare.queue_time_us":17618,"prepare.run_cpu_time_us":2034,"prepare.run_wall_time_us":46208,"raft.queue_time_us":84,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":687,"replication_time_us":50475}]]}
I20260810 07:53:08.168299 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b3e30bad905c4837820169afe8a01cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:08.170584 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3e30bad905c4837820169afe8a01cd4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:08.155865 2397 raft_consensus.cc:2243] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:08.173799 2397 raft_consensus.cc:2272] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:53:08.119398 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d4b32c5738e41c197473d44c3151c37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:08.183512 2417 tablet_replica.cc:333] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:08.184928 1047 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:08.187206 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 54a3bd8369634e4bbfdf8920454357bc (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]) successfully deleted
W20260810 07:53:08.193950 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet a89cdc44a8c64ccd83fd30855798f87b because the tablet was not found. No further retry: Not found: Tablet not found: a89cdc44a8c64ccd83fd30855798f87b
W20260810 07:53:08.194988 574 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc
W20260810 07:53:08.195770 574 leader_election.cc:343] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:08.202194 1071 tablet_bootstrap.cc:654] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:08.204525 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 2c171d3358e44ae2962ab3c4c377b0f1 because the tablet was not found. No further retry: Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
W20260810 07:53:08.205283 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 37b48c88571040f6be8d7cbd388feb58 because the tablet was not found. No further retry: Not found: Tablet not found: 37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:08.208328 859 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:08.215575 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.215830 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.219702 1064 raft_consensus.cc:359] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.225299 1064 raft_consensus.cc:385] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.225718 1064 raft_consensus.cc:740] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.229840 1064 consensus_queue.cc:260] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.216516 886 raft_consensus.cc:2468] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:08.249708 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 29) took 2112 ms (client timeout 1000 ms). Trace:
I20260810 07:53:08.269590 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 3 got 2
I20260810 07:53:08.273937 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
W20260810 07:53:08.254462 32468 rpcz_store.cc:269] 0810 07:53:06.137688 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:06.469525 (+331837us) service_pool.cc:224] Handling call
0810 07:53:08.249601 (+1780076us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:07.427218 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:07.427596 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:07.427619 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:07.427630 (+ 11us) tablet.cc:662] Decoding operations
0810 07:53:07.427893 (+ 263us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:07.427929 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:07.427951 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:07.428091 (+ 140us) tablet.cc:701] Row locks acquired
0810 07:53:07.428104 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:07.428217 (+ 113us) write_op.cc:270] Start()
0810 07:53:07.428301 (+ 84us) write_op.cc:276] Timestamp: P: 1786348387428136 usec, L: 0
0810 07:53:07.428315 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.428890 (+ 575us) log.cc:844] Serialized 533 byte log entry
0810 07:53:07.470775 (+ 41885us) op_driver.cc:464] REPLICATION: finished
0810 07:53:07.471806 (+ 1031us) write_op.cc:301] APPLY: starting
0810 07:53:07.471892 (+ 86us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:07.471986 (+ 94us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:07.472073 (+ 87us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:07.472267 (+ 194us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:07.472396 (+ 129us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:07.472480 (+ 84us) write_op.cc:312] APPLY: finished
0810 07:53:07.472668 (+ 188us) log.cc:844] Serialized 23 byte log entry
0810 07:53:07.472850 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.472909 (+ 59us) write_op.cc:454] Released schema lock
0810 07:53:07.472939 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:07.473208 (+ 269us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:07.473245 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1178,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":846,"apply.run_cpu_time_us":1354,"apply.run_wall_time_us":1354,"num_ops":1,"prepare.queue_time_us":21486,"prepare.run_cpu_time_us":1887,"prepare.run_wall_time_us":4027,"replication_time_us":42352}]]}
I20260810 07:53:08.293663 1064 ts_tablet_manager.cc:1434] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.169s user 0.007s sys 0.002s
I20260810 07:53:08.295822 1064 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:08.244617 2420 tablet.cc:1724] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260810 07:53:08.247638 855 consensus_meta.cc:287] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent flushing consensus metadata: real 0.604s user 0.003s sys 0.009s
I20260810 07:53:08.201685 2417 raft_consensus.cc:2243] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:08.194425 1047 tablet_bootstrap.cc:654] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:08.311102 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 115358us
I20260810 07:53:08.311849 2417 raft_consensus.cc:2272] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:08.314030 2417 ts_tablet_manager.cc:1916] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:08.317044 32463 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_9a4d68bb16a74d4786e2176453635dc7" } modify_external_catalogs: true
I20260810 07:53:08.322319 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c9920a33cc93419ca3284a0b009236df (table default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b]). Expected version 1 got 0
I20260810 07:53:08.324540 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet c292cb975c504ad8abf03e913140c7a9 from {username='slave'} at 127.0.0.1:43806
W20260810 07:53:08.324605 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because the tablet was not found. No further retry: Not found: Tablet not found: c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:08.342768 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.294163 2412 ts_tablet_manager.cc:1916] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:08.357409 574 leader_election.cc:343] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.323278 32469 catalog_manager.cc:5719] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:08.383651 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:08.385692 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:08.387066 32469 catalog_manager.cc:5719] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:08.388311 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:08.388747 1047 tablet_bootstrap.cc:492] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:08.389166 1047 ts_tablet_manager.cc:1403] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.204s user 0.022s sys 0.003s
I20260810 07:53:08.391665 1047 raft_consensus.cc:359] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.392323 1047 raft_consensus.cc:385] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.392616 1047 raft_consensus.cc:740] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.392655 2417 ts_tablet_manager.cc:1929] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:08.393437 1047 consensus_queue.cc:260] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.393118 2417 log.cc:1199] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:08.413465 2417 ts_tablet_manager.cc:1950] T dd22d1029ff249a0a9e05862293261ff P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:08.365718 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 410045us
I20260810 07:53:08.369976 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:53:08.370862 309 leader_election.cc:343] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:08.419305 573 leader_election.cc:304] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:08.421326 2408 raft_consensus.cc:2804] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:08.421693 2408 raft_consensus.cc:493] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:08.422020 2408 raft_consensus.cc:3060] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.423005 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet dd22d1029ff249a0a9e05862293261ff (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]) successfully deleted
W20260810 07:53:08.423930 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 because the tablet was not found. No further retry: Not found: Tablet not found: acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:08.426949 2446 tablet.cc:1724] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260810 07:53:08.429382 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 5e4b3ca46c9c44559761e25bcde3e927 because the tablet was not found. No further retry: Not found: Tablet not found: 5e4b3ca46c9c44559761e25bcde3e927
I20260810 07:53:08.433899 864 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55008 (request call id 250) took 1112 ms. Trace:
W20260810 07:53:08.431141 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 195153us
I20260810 07:53:08.434316 864 rpcz_store.cc:276] 0810 07:53:07.275075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.348058 (+ 72983us) service_pool.cc:224] Handling call
0810 07:53:07.571153 (+223095us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:07.636781 (+ 65628us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:08.058919 (+422138us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.320952 (+262033us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:08.386924 (+ 65972us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:08.203408 2397 ts_tablet_manager.cc:1916] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:08.400696 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:08.139571 380 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 0.671s user 0.006s sys 0.011s
I20260810 07:53:08.445021 2430 raft_consensus.cc:493] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:08.445552 2430 raft_consensus.cc:515] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.446251 411 raft_consensus.cc:3060] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:08.417646 308 leader_election.cc:343] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:08.448946 380 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 0.980s user 0.009s sys 0.012s
I20260810 07:53:08.439656 309 leader_election.cc:304] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.468168 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7000d25f3bfe4c99be54a0f4789cc96c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:08.470350 2382 raft_consensus.cc:2749] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:08.470566 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 100124us
I20260810 07:53:08.470974 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:08.471457 309 leader_election.cc:304] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.395779 1047 ts_tablet_manager.cc:1434] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:53:08.473829 676 raft_consensus.cc:3060] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.474184 1047 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:08.481042 792 leader_election.cc:343] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.483608 678 raft_consensus.cc:3060] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.483892 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:08.485327 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:08.488415 669 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.491261 1047 tablet_bootstrap.cc:654] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.498008 1064 tablet_bootstrap.cc:654] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.450312 2430 leader_election.cc:290] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:08.538216 2420 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:08.541708 2446 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 286) took 1104 ms. Trace:
I20260810 07:53:08.557574 309 leader_election.cc:304] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.558557 2418 raft_consensus.cc:2804] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:08.558900 2418 raft_consensus.cc:493] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:08.559207 2418 raft_consensus.cc:3060] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:08.560595 309 consensus_peers.cc:597] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:08.561218 411 raft_consensus.cc:2468] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:08.561779 2382 raft_consensus.cc:2749] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:08.563858 2408 raft_consensus.cc:515] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.566027 2408 leader_election.cc:290] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:08.566262 2426 consensus_queue.cc:1048] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:08.565446 309 leader_election.cc:304] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:08.568522 574 leader_election.cc:343] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc
I20260810 07:53:08.571189 2382 raft_consensus.cc:2804] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:08.571758 2382 raft_consensus.cc:697] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
W20260810 07:53:08.568661 309 leader_election.cc:343] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.577009 1047 tablet_bootstrap.cc:492] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:08.577574 1047 ts_tablet_manager.cc:1403] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.002s
I20260810 07:53:08.561702 2456 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:08.582445 32465 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 288) took 2542 ms. Trace:
I20260810 07:53:08.542232 2446 rpcz_store.cc:276] 0810 07:53:07.437132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.437278 (+ 146us) service_pool.cc:224] Handling call
0810 07:53:08.541624 (+1104346us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:07.477272 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:07.477630 (+ 358us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:07.477659 (+ 29us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:07.477701 (+ 42us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:07.477818 (+ 117us) alter_schema_op.cc:128] START. Timestamp: 1786348387477739
0810 07:53:07.477836 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.478330 (+ 494us) log.cc:844] Serialized 173 byte log entry
0810 07:53:08.425841 (+947511us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.426852 (+ 1011us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:08.428184 (+ 1332us) log.cc:844] Serialized 5 byte log entry
0810 07:53:08.539658 (+111474us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.540602 (+ 944us) log.cc:844] Serialized 14 byte log entry
0810 07:53:08.541416 (+ 814us) alter_schema_op.cc:73] Released schema lock
0810 07:53:08.541463 (+ 47us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":811,"mutex_wait_us":31593,"prepare.queue_time_us":274,"prepare.run_cpu_time_us":21892,"prepare.run_wall_time_us":336245,"raft.queue_time_us":50692,"raft.run_cpu_time_us":2088,"raft.run_wall_time_us":7218,"replication_time_us":947879,"thread_start_us":325432,"threads_started":3,"wal-append.queue_time_us":112342}]]}
I20260810 07:53:08.582813 32465 rpcz_store.cc:276] 0810 07:53:06.040133 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.103474 (+2063341us) service_pool.cc:224] Handling call
0810 07:53:08.316808 (+213334us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:08.344625 (+ 27817us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:08.581806 (+237181us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:08.582102 (+ 296us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:08.582379 (+ 277us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.392908 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.393390 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.393421 (+ 31us) write_op.cc:435] Acquired schema lock
0810 07:53:08.393461 (+ 40us) tablet.cc:662] Decoding operations
0810 07:53:08.393817 (+ 356us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.393877 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.393904 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.394144 (+ 240us) tablet.cc:701] Row locks acquired
0810 07:53:08.394157 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:08.394289 (+ 132us) write_op.cc:270] Start()
0810 07:53:08.394384 (+ 95us) write_op.cc:276] Timestamp: P: 1786348388394212 usec, L: 0
0810 07:53:08.394401 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.394998 (+ 597us) log.cc:844] Serialized 433 byte log entry
0810 07:53:08.514058 (+119060us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.551805 (+ 37747us) write_op.cc:301] APPLY: starting
0810 07:53:08.551872 (+ 67us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.551994 (+ 122us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.552100 (+ 106us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.552557 (+ 457us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.580291 (+ 27734us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:08.580352 (+ 61us) write_op.cc:312] APPLY: finished
0810 07:53:08.580575 (+ 223us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.581312 (+ 737us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.581501 (+ 189us) write_op.cc:454] Released schema lock
0810 07:53:08.581539 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":37520,"num_ops":2,"prepare.queue_time_us":46744,"prepare.run_cpu_time_us":2429,"prepare.run_wall_time_us":2429,"raft.queue_time_us":784,"raft.run_cpu_time_us":814,"raft.run_wall_time_us":814,"replication_time_us":119524,"thread_start_us":86567,"threads_started":2}]]}
I20260810 07:53:08.592980 367 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57540 (request call id 258) took 1117 ms. Trace:
I20260810 07:53:08.593333 367 rpcz_store.cc:276] 0810 07:53:07.474956 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.477683 (+ 2727us) service_pool.cc:224] Handling call
0810 07:53:07.673083 (+195400us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:07.711748 (+ 38665us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:08.443130 (+731382us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.588034 (+144904us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:08.592792 (+ 4758us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:08.591284 309 leader_election.cc:304] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
W20260810 07:53:08.594939 377 consensus_meta.cc:287] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent flushing consensus metadata: real 0.522s user 0.012s sys 0.002s
W20260810 07:53:08.596570 308 leader_election.cc:343] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.597609 308 leader_election.cc:304] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.581386 2382 consensus_queue.cc:237] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.598421 1047 raft_consensus.cc:359] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.599092 1047 raft_consensus.cc:385] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.599361 1047 raft_consensus.cc:740] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.584391 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe822aa30b54fb091fd62ed8b6e22ff" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:08.600024 1047 consensus_queue.cc:260] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.620004 1047 ts_tablet_manager.cc:1434] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260810 07:53:08.622078 1047 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:08.630111 1047 tablet_bootstrap.cc:654] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.637440 2456 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 285) took 1210 ms. Trace:
I20260810 07:53:08.638598 2456 rpcz_store.cc:276] 0810 07:53:07.427761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.427892 (+ 131us) service_pool.cc:224] Handling call
0810 07:53:08.637366 (+1209474us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:07.429161 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:07.429641 (+ 480us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:07.429673 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:07.429720 (+ 47us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:07.429854 (+ 134us) alter_schema_op.cc:128] START. Timestamp: 1786348387429755
0810 07:53:07.429868 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:07.430380 (+ 512us) log.cc:844] Serialized 173 byte log entry
0810 07:53:08.532487 (+1102107us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.561579 (+ 29092us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:08.580004 (+ 18425us) log.cc:844] Serialized 5 byte log entry
0810 07:53:08.629592 (+ 49588us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.630381 (+ 789us) log.cc:844] Serialized 14 byte log entry
0810 07:53:08.637193 (+ 6812us) alter_schema_op.cc:73] Released schema lock
0810 07:53:08.637236 (+ 43us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":28892,"mutex_wait_us":21126,"prepare.queue_time_us":270,"prepare.run_cpu_time_us":17472,"prepare.run_wall_time_us":405544,"raft.queue_time_us":22507,"raft.run_cpu_time_us":1456,"raft.run_wall_time_us":1808,"replication_time_us":1102470,"thread_start_us":399777,"threads_started":2,"wal-append.queue_time_us":372618}]]}
I20260810 07:53:08.650882 2418 raft_consensus.cc:515] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.661955 1047 tablet_bootstrap.cc:492] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:08.662498 1047 ts_tablet_manager.cc:1403] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.041s user 0.017s sys 0.013s
I20260810 07:53:08.662685 2418 leader_election.cc:290] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:08.663354 2418 raft_consensus.cc:2804] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:08.664022 2418 raft_consensus.cc:493] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:08.664446 2418 raft_consensus.cc:3060] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.667402 1047 raft_consensus.cc:359] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.668041 1047 raft_consensus.cc:385] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.668298 1047 raft_consensus.cc:740] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.668926 1047 consensus_queue.cc:260] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.671022 1047 ts_tablet_manager.cc:1434] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260810 07:53:08.673604 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213fa4838fe847de87091cfba97eaedc" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:08.674329 410 raft_consensus.cc:3060] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.677906 1047 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:08.690270 2460 tablet.cc:1724] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:08.691874 1047 tablet_bootstrap.cc:654] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.693958 2459 raft_consensus.cc:2749] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:08.697355 1071 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:08.697858 1071 ts_tablet_manager.cc:1403] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.571s user 0.018s sys 0.000s
W20260810 07:53:08.701326 792 leader_election.cc:343] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.701779 792 leader_election.cc:304] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.702711 1064 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:08.703171 1064 ts_tablet_manager.cc:1403] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.408s user 0.015s sys 0.000s
I20260810 07:53:08.703351 792 leader_election.cc:304] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.703100 2430 raft_consensus.cc:2749] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:08.714670 1047 tablet_bootstrap.cc:492] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:08.715168 1047 ts_tablet_manager.cc:1403] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.037s user 0.006s sys 0.003s
I20260810 07:53:08.718619 1047 raft_consensus.cc:359] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.719340 1047 raft_consensus.cc:385] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.722483 1071 raft_consensus.cc:359] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.723351 1071 raft_consensus.cc:385] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.719596 1047 raft_consensus.cc:740] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.723655 1071 raft_consensus.cc:740] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.724690 1047 consensus_queue.cc:260] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.724246 1071 consensus_queue.cc:260] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.729894 1047 ts_tablet_manager.cc:1434] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260810 07:53:08.734508 1047 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:08.737571 1071 ts_tablet_manager.cc:1434] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260810 07:53:08.738596 1071 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:08.739001 2408 raft_consensus.cc:493] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:08.739431 2408 raft_consensus.cc:515] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.742194 2468 tablet.cc:1724] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:08.742707 2408 leader_election.cc:290] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:08.744863 2375 raft_consensus.cc:493] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:08.745383 2375 raft_consensus.cc:515] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.746093 1047 tablet_bootstrap.cc:654] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.747596 2375 leader_election.cc:290] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:08.753805 2375 raft_consensus.cc:493] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:08.754416 2375 raft_consensus.cc:515] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.764632 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:08.770041 2430 raft_consensus.cc:2804] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:53:08.770226 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 66937.8us
I20260810 07:53:08.770633 2430 raft_consensus.cc:697] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:08.771528 2430 consensus_queue.cc:237] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.756496 2375 leader_election.cc:290] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:08.782829 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.783499 877 raft_consensus.cc:2468] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:08.785508 574 leader_election.cc:304] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:08.787716 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.788266 896 raft_consensus.cc:2468] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:08.789865 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:08.790436 893 raft_consensus.cc:2468] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:08.793032 2408 raft_consensus.cc:2804] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:53:08.793383 2408 raft_consensus.cc:493] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:08.793699 2408 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:08.800971 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9be900b701a45728a44bf26350a26ab" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:08.801584 877 raft_consensus.cc:3060] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.810403 573 leader_election.cc:304] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:08.811954 2334 raft_consensus.cc:2749] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:08.743646 1064 raft_consensus.cc:359] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:08.827569 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 256696us
W20260810 07:53:08.828440 308 leader_election.cc:343] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.832876 1064 raft_consensus.cc:385] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.833280 1064 raft_consensus.cc:740] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.833982 1064 consensus_queue.cc:260] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.834067 2408 raft_consensus.cc:515] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.839591 2456 tablet.cc:1724] T b6226a2377864d47b14901df39677127 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:08.840050 309 leader_election.cc:304] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:08.841632 1064 ts_tablet_manager.cc:1434] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.104s user 0.004s sys 0.002s
W20260810 07:52:54.572676 1588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:53:08.847316 2412 ts_tablet_manager.cc:1929] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260810 07:52:54.682556 1589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:53:08.847759 2412 log.cc:1199] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/3fe822aa30b54fb091fd62ed8b6e22ff
I20260810 07:53:08.848527 2412 ts_tablet_manager.cc:1950] T 3fe822aa30b54fb091fd62ed8b6e22ff P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:08.850085 573 leader_election.cc:304] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:08.850905 2407 raft_consensus.cc:2749] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:52:54.500835 1292 server_base.cc:1061] running on GCE node
I20260810 07:53:08.856594 1292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:53:08.858528 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 6a97561b5ff5490890e756ca5ff50d38
I20260810 07:53:08.860008 32464 catalog_manager.cc:6807] ReplaceTablet: tablet 6a97561b5ff5490890e756ca5ff50d38 deleted and replaced by tablet 666f36dafa92421cae47c94593ed8899
I20260810 07:53:08.860348 32464 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 290) took 2610 ms. Trace:
I20260810 07:53:08.862512 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 with delete_type TABLET_DATA_DELETED (replaced by tablet 666f36dafa92421cae47c94593ed8899) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:08.863355 309 leader_election.cc:304] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:08.827909 877 raft_consensus.cc:2468] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:08.864640 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 6a97561b5ff5490890e756ca5ff50d38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a97561b5ff5490890e756ca5ff50d38 already in progress: creating tablet
I20260810 07:53:08.862622 2426 raft_consensus.cc:2804] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:08.865087 2426 raft_consensus.cc:493] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:08.865427 2426 raft_consensus.cc:3060] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.866340 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 with delete_type TABLET_DATA_DELETED (replaced by tablet 666f36dafa92421cae47c94593ed8899) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:08.860651 32464 rpcz_store.cc:276] 0810 07:53:06.249760 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.164497 (+1914737us) service_pool.cc:224] Handling call
0810 07:53:08.860281 (+695784us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.397411 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.397911 (+ 500us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.397944 (+ 33us) write_op.cc:435] Acquired schema lock
0810 07:53:08.397963 (+ 19us) tablet.cc:662] Decoding operations
0810 07:53:08.398380 (+ 417us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.398429 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.398455 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.398713 (+ 258us) tablet.cc:701] Row locks acquired
0810 07:53:08.398731 (+ 18us) write_op.cc:260] PREPARE: finished
0810 07:53:08.398878 (+ 147us) write_op.cc:270] Start()
0810 07:53:08.398979 (+ 101us) write_op.cc:276] Timestamp: P: 1786348388398800 usec, L: 0
0810 07:53:08.399000 (+ 21us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.399569 (+ 569us) log.cc:844] Serialized 628 byte log entry
0810 07:53:08.853402 (+453833us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.855816 (+ 2414us) write_op.cc:301] APPLY: starting
0810 07:53:08.855877 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.856001 (+ 124us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.856116 (+ 115us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.856505 (+ 389us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.856753 (+ 248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:08.856809 (+ 56us) write_op.cc:312] APPLY: finished
0810 07:53:08.857013 (+ 204us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.857170 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.857236 (+ 66us) write_op.cc:454] Released schema lock
0810 07:53:08.857266 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2221,"apply.run_cpu_time_us":1679,"apply.run_wall_time_us":1680,"num_ops":2,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":2343,"prepare.run_wall_time_us":2383,"replication_time_us":454281}]]}
I20260810 07:53:08.872226 1064 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:08.864497 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 259) took 1208 ms. Trace:
I20260810 07:53:08.873706 2412 rpcz_store.cc:276] 0810 07:53:07.656231 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.656359 (+ 128us) service_pool.cc:224] Handling call
0810 07:53:07.947243 (+290884us) ts_tablet_manager.cc:1151] Deleting tablet 3fe822aa30b54fb091fd62ed8b6e22ff
0810 07:53:07.947311 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:08.509677 (+562366us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.847166 (+337489us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.864430 (+ 17264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2531,"tablet-delete.queue_time_us":233596}
I20260810 07:53:08.878561 32470 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 289) took 2728 ms. Trace:
I20260810 07:53:08.878902 32470 rpcz_store.cc:276] 0810 07:53:06.150229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.111614 (+1961385us) service_pool.cc:224] Handling call
0810 07:53:08.318436 (+206822us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:08.400308 (+ 81872us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:08.877956 (+477648us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:08.878239 (+ 283us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:08.878499 (+ 260us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.408431 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.408988 (+ 557us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.409020 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:53:08.409039 (+ 19us) tablet.cc:662] Decoding operations
0810 07:53:08.409373 (+ 334us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.409423 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.409452 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.409698 (+ 246us) tablet.cc:701] Row locks acquired
0810 07:53:08.409712 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:08.409860 (+ 148us) write_op.cc:270] Start()
0810 07:53:08.409961 (+ 101us) write_op.cc:276] Timestamp: P: 1786348388409785 usec, L: 0
0810 07:53:08.409977 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.410585 (+ 608us) log.cc:844] Serialized 433 byte log entry
0810 07:53:08.876069 (+465484us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.876308 (+ 239us) write_op.cc:301] APPLY: starting
0810 07:53:08.876362 (+ 54us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.876461 (+ 99us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.876552 (+ 91us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.876980 (+ 428us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.877109 (+ 129us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:08.877151 (+ 42us) write_op.cc:312] APPLY: finished
0810 07:53:08.877335 (+ 184us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.877498 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.877565 (+ 67us) write_op.cc:454] Released schema lock
0810 07:53:08.877598 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":98,"apply.run_cpu_time_us":1481,"apply.run_wall_time_us":1481,"num_ops":2,"prepare.queue_time_us":4878,"prepare.run_cpu_time_us":2429,"prepare.run_wall_time_us":2438,"replication_time_us":465991}]]}
I20260810 07:53:08.884820 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a97561b5ff5490890e756ca5ff50d38 with delete_type TABLET_DATA_DELETED (replaced by tablet 666f36dafa92421cae47c94593ed8899) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:08.885495 741 heartbeater.cc:344] Connected to a master server at 127.31.139.125:46845
W20260810 07:53:08.885520 32636 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad0c0 after lost signal to thread 32532
W20260810 07:53:08.886163 32636 debug-util.cc:398] Leaking SignalData structure 0x7b08000add40 after lost signal to thread 32639
I20260810 07:53:08.891146 32467 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 286) took 3062 ms. Trace:
I20260810 07:53:08.893802 32461 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 291) took 2585 ms. Trace:
I20260810 07:53:08.894121 32461 rpcz_store.cc:276] 0810 07:53:06.308536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.167914 (+1859378us) service_pool.cc:224] Handling call
0810 07:53:08.318546 (+150632us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:08.405787 (+ 87241us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:08.893189 (+487402us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:08.893454 (+ 265us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:08.893703 (+ 249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.421220 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.421687 (+ 467us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.421718 (+ 31us) write_op.cc:435] Acquired schema lock
0810 07:53:08.421735 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:08.422082 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.422129 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.422155 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.422408 (+ 253us) tablet.cc:701] Row locks acquired
0810 07:53:08.431755 (+ 9347us) write_op.cc:260] PREPARE: finished
0810 07:53:08.431918 (+ 163us) write_op.cc:270] Start()
0810 07:53:08.436671 (+ 4753us) write_op.cc:276] Timestamp: P: 1786348388431838 usec, L: 0
0810 07:53:08.436695 (+ 24us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.437382 (+ 687us) log.cc:844] Serialized 433 byte log entry
0810 07:53:08.890939 (+453557us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.891205 (+ 266us) write_op.cc:301] APPLY: starting
0810 07:53:08.891308 (+ 103us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.891410 (+ 102us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.891502 (+ 92us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.891944 (+ 442us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.892078 (+ 134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:08.892119 (+ 41us) write_op.cc:312] APPLY: finished
0810 07:53:08.892293 (+ 174us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.892906 (+ 613us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.892967 (+ 61us) write_op.cc:454] Released schema lock
0810 07:53:08.892998 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":87,"apply.run_cpu_time_us":1567,"apply.run_wall_time_us":2029,"num_ops":2,"prepare.queue_time_us":14221,"prepare.run_cpu_time_us":2463,"prepare.run_wall_time_us":16420,"replication_time_us":454154}]]}
I20260810 07:53:08.897912 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 3fe822aa30b54fb091fd62ed8b6e22ff (table default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]) successfully deleted
I20260810 07:53:08.897862 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d4c51e03eea74f81a39e48e631649307" } new_table_name: "default.Table_f2ff63903b4c44fdbc86a2d11306de7d" modify_external_catalogs: true
I20260810 07:53:08.891521 32467 rpcz_store.cc:276] 0810 07:53:05.828274 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.976812 (+2148538us) service_pool.cc:224] Handling call
0810 07:53:08.318627 (+341815us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:08.404188 (+ 85561us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:08.890529 (+486341us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:08.890818 (+ 289us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:08.891081 (+ 263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.411006 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.411494 (+ 488us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.411525 (+ 31us) write_op.cc:435] Acquired schema lock
0810 07:53:08.411540 (+ 15us) tablet.cc:662] Decoding operations
0810 07:53:08.411875 (+ 335us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.411915 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.411946 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.412178 (+ 232us) tablet.cc:701] Row locks acquired
0810 07:53:08.412198 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:08.412318 (+ 120us) write_op.cc:270] Start()
0810 07:53:08.412418 (+ 100us) write_op.cc:276] Timestamp: P: 1786348388412246 usec, L: 0
0810 07:53:08.412441 (+ 23us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.420842 (+ 8401us) log.cc:844] Serialized 433 byte log entry
0810 07:53:08.888120 (+467278us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.888400 (+ 280us) write_op.cc:301] APPLY: starting
0810 07:53:08.888460 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.888571 (+ 111us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.888669 (+ 98us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.889133 (+ 464us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.889283 (+ 150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:08.889325 (+ 42us) write_op.cc:312] APPLY: finished
0810 07:53:08.889551 (+ 226us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.890215 (+ 664us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.890299 (+ 84us) write_op.cc:454] Released schema lock
0810 07:53:08.890340 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":109,"apply.run_cpu_time_us":1648,"apply.run_wall_time_us":2147,"num_ops":2,"prepare.queue_time_us":5569,"prepare.run_cpu_time_us":2402,"prepare.run_wall_time_us":10074,"replication_time_us":475553}]]}
W20260810 07:53:08.901569 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 213fa4838fe847de87091cfba97eaedc because the tablet was not found. No further retry: Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc
I20260810 07:53:08.901610 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_870a1aff5f614bebbde025da24dedfa8" } new_table_name: "default.Table_5a9f797e42664156b0addc3bbf49ffc2" modify_external_catalogs: true
W20260810 07:53:08.886627 792 leader_election.cc:343] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:08.903276 792 leader_election.cc:304] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:08.904153 2333 raft_consensus.cc:2749] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:08.903120 32461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_3a6b61ca82b74657a1283940e4704e5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:08.906213 32461 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a6b61ca82b74657a1283940e4704e5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:08.911553 2408 leader_election.cc:290] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:08.919046 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:08.903637 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_eb0339f23b7749ccb8c8081ee9dcae9d" } modify_external_catalogs: true
W20260810 07:53:08.904366 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 because the tablet was not found. No further retry: Not found: Tablet not found: acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:08.919615 893 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:08.920038 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_eb0339f23b7749ccb8c8081ee9dcae9d" } modify_external_catalogs: true
W20260810 07:53:08.926558 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 7000d25f3bfe4c99be54a0f4789cc96c because the tablet was not found. No further retry: Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:08.928201 2412 tablet_replica.cc:333] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:08.931437 2412 raft_consensus.cc:2243] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:08.862859 410 raft_consensus.cc:2468] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:08.940590 2412 raft_consensus.cc:2272] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:08.928393 893 raft_consensus.cc:2468] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 2.
I20260810 07:53:08.943110 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:08.943637 1047 tablet_bootstrap.cc:492] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:08.944067 1047 ts_tablet_manager.cc:1403] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.212s user 0.007s sys 0.028s
I20260810 07:53:08.862118 2418 raft_consensus.cc:515] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.946662 2460 tablet.cc:1724] T b6226a2377864d47b14901df39677127 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:08.948431 574 leader_election.cc:304] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:08.949172 2481 raft_consensus.cc:2804] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:53:08.949620 2481 raft_consensus.cc:697] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:08.950397 2481 consensus_queue.cc:237] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.942493 2412 ts_tablet_manager.cc:1916] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:08.962407 1047 raft_consensus.cc:359] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:08.966460 2397 ts_tablet_manager.cc:1929] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:08.966981 2397 log.cc:1199] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:08.967365 1071 tablet_bootstrap.cc:654] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.965821 573 leader_election.cc:304] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:08.968485 2408 raft_consensus.cc:2804] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:08.969269 2408 raft_consensus.cc:697] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:08.941077 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef32f7951f8640e9a13b069816b1cdd5" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:08.973011 2408 consensus_queue.cc:237] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:08.976210 1047 raft_consensus.cc:385] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:08.976938 1047 raft_consensus.cc:740] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:08.980182 1064 tablet_bootstrap.cc:654] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:08.999207 1047 consensus_queue.cc:260] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.001533 1292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:53:09.002453 2375 raft_consensus.cc:493] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.008457 2461 raft_consensus.cc:2804] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:09.008862 2461 raft_consensus.cc:493] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:09.009176 2461 raft_consensus.cc:3060] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:08.968130 2397 ts_tablet_manager.cc:1950] T dd22d1029ff249a0a9e05862293261ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:09.010311 1292 hybrid_clock.cc:648] HybridClock initialized: now 1786348389009844 us; error 456 us; skew 500 ppm
I20260810 07:53:09.011677 1292 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:53:08.945313 2382 consensus_queue.cc:1048] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
I20260810 07:53:09.025182 1047 ts_tablet_manager.cc:1434] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.081s user 0.004s sys 0.004s
I20260810 07:53:09.028126 1064 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:09.029841 1064 ts_tablet_manager.cc:1403] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.187s user 0.013s sys 0.000s
I20260810 07:53:09.029953 1047 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.031796 2487 tablet.cc:1724] T 7a1eb27f536648c888a5acc3e940e4c0 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:09.032557 1064 raft_consensus.cc:359] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.007189 2375 raft_consensus.cc:515] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.031854 2426 raft_consensus.cc:515] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.035566 2426 leader_election.cc:290] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.035809 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 256) took 1568 ms. Trace:
I20260810 07:53:09.036954 2418 leader_election.cc:290] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.039748 2397 rpcz_store.cc:276] 0810 07:53:07.467823 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.467942 (+ 119us) service_pool.cc:224] Handling call
0810 07:53:08.032935 (+564993us) ts_tablet_manager.cc:1151] Deleting tablet dd22d1029ff249a0a9e05862293261ff
0810 07:53:08.033016 (+ 81us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:08.551209 (+518193us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:08.966236 (+415027us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.035711 (+ 69475us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121645,"tablet-delete.queue_time_us":564372,"thread_start_us":980653,"threads_started":1}
I20260810 07:53:09.043246 1064 raft_consensus.cc:385] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.043607 1064 raft_consensus.cc:740] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:09.038924 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 175649us
I20260810 07:53:09.048921 1064 consensus_queue.cc:260] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.050240 1047 tablet_bootstrap.cc:654] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.053402 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:53:09.054085 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 213217us
I20260810 07:53:09.061719 1064 ts_tablet_manager.cc:1434] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260810 07:53:09.070611 1064 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:09.048107 2412 ts_tablet_manager.cc:1929] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:09.079267 2412 log.cc:1199] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/0098aaa7be644c9885167dc1ee4929b5
I20260810 07:53:09.080235 2412 ts_tablet_manager.cc:1950] T 0098aaa7be644c9885167dc1ee4929b5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:09.083382 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 0098aaa7be644c9885167dc1ee4929b5 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]) successfully deleted
I20260810 07:53:09.089852 2412 tablet_replica.cc:333] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:09.091660 2426 consensus_queue.cc:1048] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:09.094561 2412 raft_consensus.cc:2243] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:09.095180 2412 raft_consensus.cc:2272] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:09.095258 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:09.095944 678 raft_consensus.cc:2468] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:09.097025 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9be900b701a45728a44bf26350a26ab" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:09.098101 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:09.098400 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0098aaa7be644c9885167dc1ee4929b5" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:09.099059 671 raft_consensus.cc:3060] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.099419 666 raft_consensus.cc:1240] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 LEADER]: Rejecting Update request from peer 60781ef3c8b84a8788f43d77e1bb3b19 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260810 07:53:09.098047 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:09.101089 667 raft_consensus.cc:3060] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.109045 671 raft_consensus.cc:2468] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:09.112428 308 leader_election.cc:343] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.112648 1047 tablet_bootstrap.cc:492] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:09.113181 1047 ts_tablet_manager.cc:1403] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.085s user 0.007s sys 0.009s
I20260810 07:53:09.117908 2468 tablet.cc:1724] T b6226a2377864d47b14901df39677127 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:09.119684 1064 tablet_bootstrap.cc:654] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.121233 2375 leader_election.cc:290] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.121892 667 raft_consensus.cc:2468] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:09.116088 1047 raft_consensus.cc:359] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.124155 1047 raft_consensus.cc:385] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.124485 1047 raft_consensus.cc:740] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:09.125133 308 leader_election.cc:343] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.125554 1047 consensus_queue.cc:260] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.126515 2412 ts_tablet_manager.cc:1916] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:09.127276 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82edc39a864f49f8be60e4b36ce3d59c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:09.133519 893 raft_consensus.cc:3060] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.129168 309 leader_election.cc:304] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:09.089237 2418 consensus_queue.cc:1048] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260810 07:53:09.141587 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c292cb975c504ad8abf03e913140c7a9
W20260810 07:53:09.140664 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4a46a037cae4f599398b289bb6a2d84 consensus state has no leader
I20260810 07:53:09.150451 1047 ts_tablet_manager.cc:1434] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260810 07:53:09.153867 32468 catalog_manager.cc:6807] ReplaceTablet: tablet c292cb975c504ad8abf03e913140c7a9 deleted and replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64
I20260810 07:53:09.154277 32468 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 292) took 2584 ms. Trace:
I20260810 07:53:09.156782 1047 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.146384 32470 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 294) took 1551 ms. Trace:
I20260810 07:53:09.116531 2461 raft_consensus.cc:515] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.162817 2461 leader_election.cc:290] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:09.163573 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet f4a46a037cae4f599398b289bb6a2d84 (table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f4a46a037cae4f599398b289bb6a2d84 consensus state has no leader
I20260810 07:53:09.167415 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 77fa93c49cb54e5fa03034eda6fc2300
I20260810 07:53:09.169451 32464 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 293) took 2492 ms. Trace:
I20260810 07:53:09.171535 864 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:09.171980 855 tablet_service.cc:1558] Processing DeleteTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:09.142524 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet dd22d1029ff249a0a9e05862293261ff (table default.Table_aa23aa2f7ea64c1387df86e35a66e368 [id=11ed9203f80f4a148c05c19d70d6990c]) successfully deleted
I20260810 07:53:09.169790 32464 rpcz_store.cc:276] 0810 07:53:06.677511 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.875469 (+2197958us) service_pool.cc:224] Handling call
0810 07:53:08.903544 (+ 28075us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c4d6 00007f30cf424b34 00007f30cf424ac2 00007f30cf4249f1 00007f30cf42496c 00007f30cf423c22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:08.920505 (+ 16961us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:08.926230 (+ 5725us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:08.926368 (+ 138us) catalog_manager.cc:2850] Locking tablets
0810 07:53:08.927014 (+ 646us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:09.166873 (+239859us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:09.167150 (+ 277us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:09.167291 (+ 141us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:09.169370 (+ 2079us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.996643 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.997202 (+ 559us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.997238 (+ 36us) write_op.cc:435] Acquired schema lock
0810 07:53:08.997263 (+ 25us) tablet.cc:662] Decoding operations
0810 07:53:08.997674 (+ 411us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.997719 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.997752 (+ 33us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.998089 (+ 337us) tablet.cc:701] Row locks acquired
0810 07:53:08.998112 (+ 23us) write_op.cc:260] PREPARE: finished
0810 07:53:08.998263 (+ 151us) write_op.cc:270] Start()
0810 07:53:08.998429 (+ 166us) write_op.cc:276] Timestamp: P: 1786348388998163 usec, L: 0
0810 07:53:08.998452 (+ 23us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.999088 (+ 636us) log.cc:844] Serialized 731 byte log entry
0810 07:53:09.161862 (+162774us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.164830 (+ 2968us) write_op.cc:301] APPLY: starting
0810 07:53:09.164900 (+ 70us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.165035 (+ 135us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.165183 (+ 148us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.165599 (+ 416us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.165817 (+ 218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:09.165887 (+ 70us) write_op.cc:312] APPLY: finished
0810 07:53:09.166160 (+ 273us) log.cc:844] Serialized 29 byte log entry
0810 07:53:09.166409 (+ 249us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.166512 (+ 103us) write_op.cc:454] Released schema lock
0810 07:53:09.166559 (+ 47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":62746880,"child_traces":[["op",{"apply.queue_time_us":2706,"num_ops":2,"prepare.queue_time_us":43376,"prepare.run_cpu_time_us":2711,"prepare.run_wall_time_us":2714,"replication_time_us":163301,"spinlock_wait_cycles":2816}]]}
I20260810 07:53:09.174340 622 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.175133 2417 tablet_replica.cc:333] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:09.175974 2417 raft_consensus.cc:2243] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:09.176090 1064 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:09.176532 1064 ts_tablet_manager.cc:1403] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.000s
I20260810 07:53:09.176966 2417 raft_consensus.cc:2272] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:09.178803 1071 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:09.179226 1071 ts_tablet_manager.cc:1403] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.441s user 0.009s sys 0.007s
I20260810 07:53:09.179219 2417 ts_tablet_manager.cc:1916] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:09.179170 1064 raft_consensus.cc:359] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.179839 1064 raft_consensus.cc:385] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.180094 1064 raft_consensus.cc:740] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.181296 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.181527 893 raft_consensus.cc:2468] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:09.160669 32470 rpcz_store.cc:276] 0810 07:53:07.594400 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.884203 (+1289803us) service_pool.cc:224] Handling call
0810 07:53:08.897746 (+ 13543us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c846 00007f30cf429634 00007f30cf4295c2 00007f30cf4294f1 00007f30cf42946c 00007f30cf428722 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:08.898563 (+ 817us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:08.899854 (+ 1291us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:08.900442 (+ 588us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:08.901485 (+ 1043us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:09.140069 (+238584us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:09.146026 (+ 5957us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.933963 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.934555 (+ 592us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.934597 (+ 42us) write_op.cc:435] Acquired schema lock
0810 07:53:08.934620 (+ 23us) tablet.cc:662] Decoding operations
0810 07:53:08.934989 (+ 369us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.935046 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.935081 (+ 35us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:08.935308 (+ 227us) tablet.cc:701] Row locks acquired
0810 07:53:08.935331 (+ 23us) write_op.cc:260] PREPARE: finished
0810 07:53:08.935488 (+ 157us) write_op.cc:270] Start()
0810 07:53:08.935584 (+ 96us) write_op.cc:276] Timestamp: P: 1786348388935404 usec, L: 0
0810 07:53:08.935606 (+ 22us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.936215 (+ 609us) log.cc:844] Serialized 405 byte log entry
0810 07:53:09.136564 (+200349us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.136890 (+ 326us) write_op.cc:301] APPLY: starting
0810 07:53:09.136973 (+ 83us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.137094 (+ 121us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.137179 (+ 85us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.137358 (+ 179us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.137487 (+ 129us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:09.137530 (+ 43us) write_op.cc:312] APPLY: finished
0810 07:53:09.137716 (+ 186us) log.cc:844] Serialized 23 byte log entry
0810 07:53:09.137880 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.137939 (+ 59us) write_op.cc:454] Released schema lock
0810 07:53:09.137974 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:09.138201 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.138228 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27933952,"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":1326,"apply.run_wall_time_us":1326,"num_ops":1,"prepare.queue_time_us":30989,"prepare.run_cpu_time_us":2523,"prepare.run_wall_time_us":2528,"replication_time_us":200839,"spinlock_wait_cycles":5376}]]}
I20260810 07:53:09.184518 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e5631a682bd14d889ec772c94d4a5c5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:09.185359 1292 webserver.cc:533] Webserver started at http://127.31.139.126:38367/ using document root <none> and password file <none>
I20260810 07:53:09.186899 375 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:09.187952 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 77fa93c49cb54e5fa03034eda6fc2300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:09.191057 1292 fs_manager.cc:362] Metadata directory not provided
I20260810 07:53:09.188683 1047 tablet_bootstrap.cc:654] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:09.204653 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5631a682bd14d889ec772c94d4a5c5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:09.160027 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 3 got 2
I20260810 07:53:09.205704 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:09.205622 1064 consensus_queue.cc:260] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.207805 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c9920a33cc93419ca3284a0b009236df (table default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b]). Expected version 1 got 0
I20260810 07:53:09.208778 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:09.209444 2430 raft_consensus.cc:493] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.154567 32468 rpcz_store.cc:276] 0810 07:53:06.569886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.324434 (+1754548us) service_pool.cc:224] Handling call
0810 07:53:09.154209 (+829775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.937899 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.938437 (+ 538us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.938469 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:53:08.938483 (+ 14us) tablet.cc:662] Decoding operations
0810 07:53:08.938853 (+ 370us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.938898 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.938928 (+ 30us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.939176 (+ 248us) tablet.cc:701] Row locks acquired
0810 07:53:08.939196 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:08.939336 (+ 140us) write_op.cc:270] Start()
0810 07:53:08.939425 (+ 89us) write_op.cc:276] Timestamp: P: 1786348388939258 usec, L: 0
0810 07:53:08.939452 (+ 27us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.940222 (+ 770us) log.cc:844] Serialized 662 byte log entry
0810 07:53:09.136872 (+196650us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.138483 (+ 1611us) write_op.cc:301] APPLY: starting
0810 07:53:09.138531 (+ 48us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.138639 (+ 108us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.138748 (+ 109us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.139157 (+ 409us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.139375 (+ 218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:09.139424 (+ 49us) write_op.cc:312] APPLY: finished
0810 07:53:09.139602 (+ 178us) log.cc:844] Serialized 29 byte log entry
0810 07:53:09.139769 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.139839 (+ 70us) write_op.cc:454] Released schema lock
0810 07:53:09.139871 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1467,"mutex_wait_us":22497,"num_ops":2,"prepare.queue_time_us":22661,"prepare.run_cpu_time_us":2441,"prepare.run_wall_time_us":2581,"replication_time_us":197319,"threads_started":1}]]}
I20260810 07:53:09.210026 2430 raft_consensus.cc:515] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.210513 1071 raft_consensus.cc:359] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.216609 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_30f89a33906a42b9bd617cd1a8d2eaae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:09.218959 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30f89a33906a42b9bd617cd1a8d2eaae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:09.220266 1292 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:53:09.224509 2430 leader_election.cc:290] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.232774 1064 ts_tablet_manager.cc:1434] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
W20260810 07:53:09.217018 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 87017.9us
I20260810 07:53:09.233637 308 leader_election.cc:304] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:09.234612 2382 raft_consensus.cc:2804] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.235074 2382 raft_consensus.cc:697] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:09.236116 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:09.235877 2382 consensus_queue.cc:237] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.238030 32465 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:09.239619 308 leader_election.cc:304] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:09.240306 2418 raft_consensus.cc:2804] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.240764 2418 raft_consensus.cc:697] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:09.241386 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:09.225067 2461 consensus_queue.cc:1059] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:09.241674 2418 consensus_queue.cc:237] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.242074 671 raft_consensus.cc:2468] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:09.244187 2461 consensus_queue.cc:1243] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Peer b5bfa873accc428ab48ac19e1207f88d log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260810 07:53:09.246021 2494 raft_consensus.cc:3055] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Stepping down as leader of term 1
I20260810 07:53:09.246352 2494 raft_consensus.cc:740] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:09.247166 2494 consensus_queue.cc:260] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.248189 2494 raft_consensus.cc:3060] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:09.250468 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:09.254823 792 leader_election.cc:343] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:09.267089 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 84254.7us
I20260810 07:53:09.270687 1064 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:09.271620 1047 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:09.272213 1047 ts_tablet_manager.cc:1403] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.116s user 0.023s sys 0.011s
I20260810 07:53:09.272385 308 leader_election.cc:304] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:09.273679 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:09.276467 1047 raft_consensus.cc:359] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.280763 1047 raft_consensus.cc:385] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.281046 1047 raft_consensus.cc:740] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.281674 1047 consensus_queue.cc:260] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.282470 1071 raft_consensus.cc:385] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.282791 1071 raft_consensus.cc:740] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.283469 1071 consensus_queue.cc:260] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.285002 2408 raft_consensus.cc:493] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.285542 2408 raft_consensus.cc:515] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.287695 2408 leader_election.cc:290] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:09.290020 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:09.290906 893 raft_consensus.cc:2393] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:09.293020 1047 ts_tablet_manager.cc:1434] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.020s user 0.001s sys 0.005s
I20260810 07:53:09.293921 1047 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:09.304519 308 leader_election.cc:343] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:09.304661 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 175567us
I20260810 07:53:09.310467 1047 tablet_bootstrap.cc:654] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.306365 2382 raft_consensus.cc:2804] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:09.311646 2382 raft_consensus.cc:493] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:09.312238 2382 raft_consensus.cc:3060] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.306649 2461 raft_consensus.cc:2804] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.313717 2461 raft_consensus.cc:697] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:09.314628 2461 consensus_queue.cc:237] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.316687 1071 ts_tablet_manager.cc:1434] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.137s user 0.006s sys 0.001s
I20260810 07:53:09.320067 1064 tablet_bootstrap.cc:654] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.318336 2412 ts_tablet_manager.cc:1929] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:09.320968 2412 log.cc:1199] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:09.321967 2412 ts_tablet_manager.cc:1950] T 7eeabaae541548afab4a3ff5ab388035 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:09.323773 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c5e309389d402092f59c801923854a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.325662 573 leader_election.cc:304] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:09.326433 2408 raft_consensus.cc:3060] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.334296 2408 raft_consensus.cc:2749] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:09.337507 309 leader_election.cc:343] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:53:09.337785 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 271) took 1192 ms. Trace:
I20260810 07:53:09.338263 2412 rpcz_store.cc:276] 0810 07:53:08.145271 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.145472 (+ 201us) service_pool.cc:224] Handling call
0810 07:53:09.089128 (+943656us) ts_tablet_manager.cc:1151] Deleting tablet 7eeabaae541548afab4a3ff5ab388035
0810 07:53:09.089632 (+ 504us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:09.196226 (+106594us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.318147 (+121921us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.337680 (+ 19533us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":899359}
I20260810 07:53:09.339811 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 7eeabaae541548afab4a3ff5ab388035 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
I20260810 07:53:09.339713 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:09.340356 893 raft_consensus.cc:2468] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:09.342869 893 raft_consensus.cc:1240] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Rejecting Update request from peer 60781ef3c8b84a8788f43d77e1bb3b19 for earlier term 1. Current term is 2. Ops: []
I20260810 07:53:09.342933 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 3 got 2
I20260810 07:53:09.343833 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c9920a33cc93419ca3284a0b009236df (table default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b]). Expected version 1 got 0
I20260810 07:53:09.344121 1064 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:09.344672 1064 ts_tablet_manager.cc:1403] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.000s
I20260810 07:53:09.346118 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:09.346738 893 raft_consensus.cc:3060] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.344563 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:09.352097 2412 tablet_replica.cc:333] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:09.354961 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:09.367161 893 raft_consensus.cc:2468] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:09.371908 1064 raft_consensus.cc:359] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.372892 1064 raft_consensus.cc:385] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.373247 1064 raft_consensus.cc:740] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.376137 2430 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.377075 1047 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:09.376847 2430 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.377583 1047 ts_tablet_manager.cc:1403] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.084s user 0.026s sys 0.035s
I20260810 07:53:09.380906 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:09.380861 1047 raft_consensus.cc:359] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.381539 1047 raft_consensus.cc:385] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.381840 1047 raft_consensus.cc:740] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:09.382503 792 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.382596 1047 consensus_queue.cc:260] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.383126 2430 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.384611 1047 ts_tablet_manager.cc:1434] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260810 07:53:09.385799 1047 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.386507 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.132637 2397 tablet_replica.cc:333] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:09.394425 1047 tablet_bootstrap.cc:654] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:09.401589 2514 log.cc:927] Time spent T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.141s user 0.001s sys 0.000s
I20260810 07:53:09.403330 2408 raft_consensus.cc:493] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.403937 2408 raft_consensus.cc:515] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.408139 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:09.411051 2408 leader_election.cc:290] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:09.414866 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5c3bed1f2a42b1bbf18b497555eb7a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.426110 414 raft_consensus.cc:2393] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:09.429422 573 leader_election.cc:304] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:09.430701 2481 raft_consensus.cc:3060] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.433894 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:09.438377 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.439666 1064 consensus_queue.cc:260] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.454792 1064 ts_tablet_manager.cc:1434] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.110s user 0.006s sys 0.000s
I20260810 07:53:09.468564 1292 fs_manager.cc:714] Time spent opening directory manager: real 0.169s user 0.004s sys 0.006s
I20260810 07:53:09.470602 2481 raft_consensus.cc:2749] T 2b5c3bed1f2a42b1bbf18b497555eb7a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:09.472106 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:09.420672 2382 raft_consensus.cc:515] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.450268 1071 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:09.477156 2412 raft_consensus.cc:2243] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:09.477708 2412 raft_consensus.cc:2272] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:09.480330 2412 ts_tablet_manager.cc:1916] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:09.482393 1071 tablet_bootstrap.cc:654] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.488826 2382 leader_election.cc:290] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.489823 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:09.490386 671 raft_consensus.cc:3060] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.494875 2397 raft_consensus.cc:2243] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:09.495945 2397 raft_consensus.cc:2272] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:09.500330 2417 ts_tablet_manager.cc:1929] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 07:53:09.503206 792 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.503774 792 leader_election.cc:304] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:09.505940 2430 raft_consensus.cc:2749] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:09.532877 2397 ts_tablet_manager.cc:1916] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:09.534150 1064 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:09.537115 309 leader_election.cc:304] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:09.541726 792 leader_election.cc:343] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.542454 792 leader_election.cc:304] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:09.544109 2418 raft_consensus.cc:2804] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.544998 2418 raft_consensus.cc:697] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:09.548233 2418 consensus_queue.cc:237] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.561084 2333 raft_consensus.cc:2749] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:09.569590 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:09.570355 888 raft_consensus.cc:3060] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.587792 888 raft_consensus.cc:2468] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:09.628994 2382 raft_consensus.cc:493] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.629482 2382 raft_consensus.cc:515] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.638681 2382 leader_election.cc:290] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:09.644872 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 31) took 2053 ms (client timeout 1000 ms). Trace:
W20260810 07:53:09.646111 32462 rpcz_store.cc:269] 0810 07:53:07.591421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.961387 (+369966us) service_pool.cc:224] Handling call
0810 07:53:09.644768 (+1683381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43491,"reported_tablets":26}
I20260810 07:53:09.659708 1071 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:09.575439 1047 tablet_bootstrap.cc:492] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:09.660849 1047 ts_tablet_manager.cc:1403] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.275s user 0.056s sys 0.080s
I20260810 07:53:09.500864 2417 log.cc:1199] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:09.663934 2417 ts_tablet_manager.cc:1950] T c292cb975c504ad8abf03e913140c7a9 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:09.614367 619 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.641994 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:09.608337 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.588678 2481 raft_consensus.cc:493] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.669384 2481 raft_consensus.cc:515] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.671751 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]) successfully deleted
W20260810 07:53:09.672753 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 77fa93c49cb54e5fa03034eda6fc2300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77fa93c49cb54e5fa03034eda6fc2300 already in progress: creating tablet
I20260810 07:53:09.605988 627 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.604025 622 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.674244 1047 raft_consensus.cc:359] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.675741 1047 raft_consensus.cc:385] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.676025 1047 raft_consensus.cc:740] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.675860 2481 leader_election.cc:290] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:09.676755 1047 consensus_queue.cc:260] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:09.678486 2537 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:53:09.681516 1047 ts_tablet_manager.cc:1434] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
I20260810 07:53:09.685222 1292 fs_manager.cc:730] Time spent opening block manager: real 0.162s user 0.004s sys 0.016s
I20260810 07:53:09.684034 1047 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.688082 671 raft_consensus.cc:2468] T f0606ccac5d04aa89d804cf9168018b8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:09.668483 308 leader_election.cc:343] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.690379 1071 ts_tablet_manager.cc:1403] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.240s user 0.008s sys 0.010s
I20260810 07:53:09.690629 308 leader_election.cc:304] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:09.685829 1292 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
uuid: "72fde54f23f34224ac756f11924c9531"
format_stamp: "Formatted at 2026-08-10 07:52:31 on dist-test-slave-x5fp"
I20260810 07:53:09.685758 2430 raft_consensus.cc:493] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.700796 2430 raft_consensus.cc:515] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.702160 1292 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260810 07:53:09.670539 309 consensus_peers.cc:597] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:09.682909 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.703409 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:09.673568 32450 catalog_manager.cc:5123] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): alter failed for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]),no further retry: Not found: Tablet not found: acc89d15b4d544b7bc8abc12a4c6d3e5
W20260810 07:53:09.717684 574 leader_election.cc:343] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.718762 1047 tablet_bootstrap.cc:654] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.721223 2430 leader_election.cc:290] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:09.726056 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 3 got 2
I20260810 07:53:09.737604 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]). Expected version 1 got 0
I20260810 07:53:09.738555 32460 catalog_manager.cc:5719] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:09.740162 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:09.741457 1047 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:09.741401 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ab03facf8477eada5d1ac3ec7f5ef" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:09.743330 792 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.741546 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_890c4de822874851bd58fd49b7b992ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:09.766656 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_890c4de822874851bd58fd49b7b992ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:09.744078 32470 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c5971a0e5bac4dbd907990ca708bd38c" } modify_external_catalogs: true
I20260810 07:53:09.771304 32470 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c5971a0e5bac4dbd907990ca708bd38c" } modify_external_catalogs: true
I20260810 07:53:09.735355 32468 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_d73cd38f1d774b448a86e7e8f5b10242" } new_table_name: "default.Table_6d66b8948fe749769480f2a7c912be17" modify_external_catalogs: true
I20260810 07:53:09.787424 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:09.793684 32470 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet e6800738970d47dd8e7cba68945042aa
I20260810 07:53:09.796160 622 tablet_service.cc:1558] Processing DeleteTablet for tablet e6800738970d47dd8e7cba68945042aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:09 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:09.799103 855 tablet_service.cc:1558] Processing DeleteTablet for tablet e6800738970d47dd8e7cba68945042aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:09 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:09.789551 32460 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:09.829092 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b76493904a5946c7b069e14e1fbdd9af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:09.839617 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:09.840423 893 raft_consensus.cc:2468] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:09.843508 1071 raft_consensus.cc:359] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.844255 1071 raft_consensus.cc:385] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.741978 1047 ts_tablet_manager.cc:1403] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.005s
I20260810 07:53:09.845163 309 leader_election.cc:304] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:09.847435 1047 raft_consensus.cc:359] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.848097 1047 raft_consensus.cc:385] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.848392 1047 raft_consensus.cc:740] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.709142 2382 raft_consensus.cc:2804] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.849609 2417 tablet_replica.cc:333] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:53:09.850075 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because the tablet was not found. No further retry: Not found: Tablet not found: 54a3bd8369634e4bbfdf8920454357bc
I20260810 07:53:09.850684 2417 raft_consensus.cc:2243] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:53:09.848088 574 consensus_peers.cc:597] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:09.709709 415 raft_consensus.cc:2468] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:09.860736 2430 consensus_queue.cc:1048] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:09.849655 2459 raft_consensus.cc:2804] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:09.867939 2459 raft_consensus.cc:493] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:09.868319 2459 raft_consensus.cc:3060] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.869716 1064 tablet_bootstrap.cc:654] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:09.851814 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ab03facf8477eada5d1ac3ec7f5ef" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.851347 2417 raft_consensus.cc:2272] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:09.879386 375 tablet_service.cc:1558] Processing DeleteTablet for tablet e6800738970d47dd8e7cba68945042aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:09 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:09.883359 1047 consensus_queue.cc:260] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.886260 1047 ts_tablet_manager.cc:1434] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260810 07:53:09.887475 1047 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.888301 2412 ts_tablet_manager.cc:1929] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:09.888835 2412 log.cc:1199] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/45c5e309389d402092f59c801923854a
I20260810 07:53:09.890918 2426 consensus_queue.cc:1048] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:09.844524 1071 raft_consensus.cc:740] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.894897 1071 consensus_queue.cc:260] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:09.905045 792 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.905534 792 leader_election.cc:304] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:09.907022 2430 raft_consensus.cc:2749] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:09.906754 2333 consensus_queue.cc:1048] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:09.905107 2412 ts_tablet_manager.cc:1950] T 45c5e309389d402092f59c801923854a P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:09.910712 1071 ts_tablet_manager.cc:1434] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.220s user 0.006s sys 0.001s
I20260810 07:53:09.912423 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 272) took 1704 ms. Trace:
I20260810 07:53:09.913239 893 raft_consensus.cc:1275] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260810 07:53:09.914974 1064 tablet_bootstrap.cc:492] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:09.915452 1064 ts_tablet_manager.cc:1403] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.382s user 0.010s sys 0.005s
I20260810 07:53:09.918277 1064 raft_consensus.cc:359] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.919050 1064 raft_consensus.cc:385] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:09.919370 1064 raft_consensus.cc:740] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:09.919209 2459 raft_consensus.cc:515] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.920284 1064 consensus_queue.cc:260] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.898195 2540 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:09.921324 1071 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:09.921325 2540 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.923570 2540 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:09.898756 573 leader_election.cc:304] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:09.925897 2408 raft_consensus.cc:2804] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:09.926272 2408 raft_consensus.cc:493] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:09.926607 2408 raft_consensus.cc:3060] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.933632 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:09.933920 2540 consensus_queue.cc:1048] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:09.936290 573 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:09.939335 2408 raft_consensus.cc:515] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:09.941869 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:09.942360 411 raft_consensus.cc:3060] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.950809 411 raft_consensus.cc:2468] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:09.952015 573 leader_election.cc:304] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:09.952692 2417 ts_tablet_manager.cc:1916] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:09.952865 2481 raft_consensus.cc:2804] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:09.953372 2481 raft_consensus.cc:697] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:09.954218 2481 consensus_queue.cc:237] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:09.966482 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b76493904a5946c7b069e14e1fbdd9af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:09.974781 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:09.974936 2408 leader_election.cc:290] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:10.009509 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:10.010752 308 leader_election.cc:343] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:10.014568 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:10.015267 888 raft_consensus.cc:3060] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:09.912895 2412 rpcz_store.cc:276] 0810 07:53:08.207601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.208157 (+ 556us) service_pool.cc:224] Handling call
0810 07:53:09.351849 (+1143692us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:09.351924 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:09.615350 (+263426us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.888137 (+272787us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.912361 (+ 24224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123891,"tablet-delete.queue_time_us":1141277}
I20260810 07:53:10.020632 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 273) took 1154 ms. Trace:
I20260810 07:53:10.021188 2412 rpcz_store.cc:276] 0810 07:53:08.866047 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.866193 (+ 146us) service_pool.cc:224] Handling call
0810 07:53:10.020026 (+1153833us) ts_tablet_manager.cc:1151] Deleting tablet 6a97561b5ff5490890e756ca5ff50d38
0810 07:53:10.020072 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:10.020552 (+ 480us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1153186}
I20260810 07:53:10.024050 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 45c5e309389d402092f59c801923854a (table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]) successfully deleted
W20260810 07:53:10.024959 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 6a97561b5ff5490890e756ca5ff50d38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a97561b5ff5490890e756ca5ff50d38 already in progress: creating tablet
I20260810 07:53:10.031710 2382 raft_consensus.cc:697] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:10.033548 2382 consensus_queue.cc:237] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.038757 2459 leader_election.cc:290] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:10.045349 1071 tablet_bootstrap.cc:654] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:09.989965 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 83300.9us
I20260810 07:53:10.055092 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f517aea30ffe40f2922cdae2d9712eb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:10.055424 2481 consensus_queue.cc:1048] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:09.980639 32464 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:10.074510 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:10.075356 896 raft_consensus.cc:3060] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:10.075451 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f517aea30ffe40f2922cdae2d9712eb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:10.083673 896 raft_consensus.cc:2468] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:10.089028 1047 tablet_bootstrap.cc:654] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:10.091213 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:10.094560 574 consensus_peers.cc:597] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:10.096278 1064 ts_tablet_manager.cc:1434] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.181s user 0.006s sys 0.000s
I20260810 07:53:10.118665 888 raft_consensus.cc:2468] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:10.133508 574 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:10.134022 574 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:10.134754 2408 raft_consensus.cc:2749] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:10.134466 32462 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_dc514c4924df483b859282812b30bb1e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:10.136003 32462 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc514c4924df483b859282812b30bb1e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:10.136399 1047 tablet_bootstrap.cc:492] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:10.136838 1047 ts_tablet_manager.cc:1403] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.250s user 0.015s sys 0.005s
I20260810 07:53:10.137373 2412 tablet_replica.cc:333] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:10.138307 2412 raft_consensus.cc:2243] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:10.138878 2412 raft_consensus.cc:2272] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:10.142226 2412 ts_tablet_manager.cc:1916] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:10.140566 1047 raft_consensus.cc:359] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.148509 1047 raft_consensus.cc:385] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:10.161405 1047 raft_consensus.cc:740] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:10.168335 1064 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:10.163781 1047 consensus_queue.cc:260] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.181993 1292 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:53:10.184517 1292 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:53:10.186254 1292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:53:10.193149 1047 ts_tablet_manager.cc:1434] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.056s user 0.007s sys 0.000s
I20260810 07:53:10.197278 1047 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:09.928509 2397 ts_tablet_manager.cc:1929] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:10.114091 2552 raft_consensus.cc:493] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:10.132309 309 leader_election.cc:304] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:10.201870 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 69681.9us
I20260810 07:53:10.214941 1047 tablet_bootstrap.cc:654] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:10.193600 414 raft_consensus.cc:1275] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Refusing update from remote peer b5bfa873accc428ab48ac19e1207f88d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260810 07:53:10.229033 414 pending_rounds.cc:85] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting all ops after (but not including) 0
I20260810 07:53:10.233160 2552 raft_consensus.cc:515] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.241349 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:10.237649 414 pending_rounds.cc:107] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260810 07:53:10.242063 414 raft_consensus.cc:2889] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260810 07:53:10.234891 2459 raft_consensus.cc:2804] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:10.245293 2375 consensus_queue.cc:1048] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:10.156440 2417 ts_tablet_manager.cc:1929] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:10.247360 2459 raft_consensus.cc:697] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:10.247913 2417 log.cc:1199] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:10.249500 2417 ts_tablet_manager.cc:1950] T 7000d25f3bfe4c99be54a0f4789cc96c P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:10.249393 2552 leader_election.cc:290] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:10.250612 2459 consensus_queue.cc:237] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.242424 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_71d864751edf4f7786f4aea1d9e5e8ad" } modify_external_catalogs: true
I20260810 07:53:10.253017 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_71d864751edf4f7786f4aea1d9e5e8ad" } modify_external_catalogs: true
I20260810 07:53:10.218787 2397 log.cc:1199] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/54a3bd8369634e4bbfdf8920454357bc
I20260810 07:53:10.258571 2397 ts_tablet_manager.cc:1950] T 54a3bd8369634e4bbfdf8920454357bc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:10.251473 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:10.256466 2408 consensus_queue.cc:1048] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:10.260149 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 257) took 2785 ms. Trace:
I20260810 07:53:10.262087 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:10.264168 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]). Expected version 1 got 0
I20260810 07:53:10.274900 2417 tablet_replica.cc:333] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:10.265767 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 7000d25f3bfe4c99be54a0f4789cc96c (table default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]) successfully deleted
I20260810 07:53:10.276520 32470 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:10.279462 32470 catalog_manager.cc:5719] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:10.284093 2417 raft_consensus.cc:2243] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:10.285512 2417 raft_consensus.cc:2272] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:10.310978 2417 ts_tablet_manager.cc:1916] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:10.308686 888 raft_consensus.cc:2468] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:10.381488 2417 ts_tablet_manager.cc:1929] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:10.382025 2417 log.cc:1199] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:10.382905 2417 ts_tablet_manager.cc:1950] T 7eeabaae541548afab4a3ff5ab388035 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:10.385851 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 7eeabaae541548afab4a3ff5ab388035 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
W20260810 07:53:10.386725 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet dd22d1029ff249a0a9e05862293261ff because the tablet was not found. No further retry: Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff
W20260810 07:53:10.392015 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet f4a46a037cae4f599398b289bb6a2d84 (table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f4a46a037cae4f599398b289bb6a2d84 consensus state has no leader
W20260810 07:53:10.394171 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet e6800738970d47dd8e7cba68945042aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6800738970d47dd8e7cba68945042aa already in progress: creating tablet
I20260810 07:53:10.407222 1047 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:10.407728 1047 ts_tablet_manager.cc:1403] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.211s user 0.014s sys 0.013s
I20260810 07:53:10.412019 1047 raft_consensus.cc:359] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.413057 1047 raft_consensus.cc:385] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:10.413398 1047 raft_consensus.cc:740] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:10.414242 1047 consensus_queue.cc:260] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.416872 1047 ts_tablet_manager.cc:1434] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260810 07:53:10.417830 1047 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:10.295173 2583 tablet.cc:1724] T fc12205a097c496cba5b777b98f27914 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:10.292912 2426 consensus_queue.cc:1048] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.132s
I20260810 07:53:10.260671 2397 rpcz_store.cc:276] 0810 07:53:07.474615 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.536899 (+ 62284us) service_pool.cc:224] Handling call
0810 07:53:09.128121 (+1591222us) ts_tablet_manager.cc:1151] Deleting tablet 54a3bd8369634e4bbfdf8920454357bc
0810 07:53:09.132397 (+ 4276us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800000f98. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c64327981 00007f9c64467521 00007f9c64467b52 00007f9c6d3d51d9 00007f9c6c5ed4e8 00007f9c6c6b3aa9 00007f9c6c6b40a3 00007f9c6c6e50e0 00007f9c6c6cb64d 00007f9c6c6cb5f9 00007f9c6c6cb589
0810 07:53:09.132478 (+ 81us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:09.859593 (+727115us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:09.928348 (+ 68755us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:10.260002 (+331654us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98252,"spinlock_wait_cycles":4796160,"tablet-delete.queue_time_us":1590187}
I20260810 07:53:10.430467 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 54a3bd8369634e4bbfdf8920454357bc (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]) successfully deleted
I20260810 07:53:10.430495 2397 tablet_replica.cc:333] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:10.431744 2397 raft_consensus.cc:2243] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:10.298642 2552 consensus_queue.cc:1048] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.135s
W20260810 07:53:10.263119 308 leader_election.cc:343] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:10.436662 309 leader_election.cc:304] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:10.437489 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 114562us
I20260810 07:53:10.440606 2459 raft_consensus.cc:2804] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:10.443018 2459 raft_consensus.cc:493] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:10.443444 2459 raft_consensus.cc:3060] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:10.452694 2408 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:10.453248 2408 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.455510 2408 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:10.272353 348 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:57532 (request call id 40) took 1115 ms. Trace:
I20260810 07:53:10.458079 348 rpcz_store.cc:276] 0810 07:53:09.139152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.142826 (+ 3674us) service_pool.cc:224] Handling call
0810 07:53:09.398012 (+255186us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000e1e28. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b9c3b41 00007f9c6ba4fd1c 00007f9c6c66d4c0 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:09.488398 (+ 90386us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b54001306d4. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b56d951 00007f9c6b5fe2d8 00007f9c6c66d4ed 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:09.599441 (+111043us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54001301d4. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b56d951 00007f9c6b5fe2d8 00007f9c6c66d4ed 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:09.939884 (+340443us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54000857d4. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b56d951 00007f9c6b5fe2d8 00007f9c6c66d4ed 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:10.200679 (+260795us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248573696}
I20260810 07:53:10.463050 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:10.462141 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 199033us
W20260810 07:53:10.469187 573 leader_election.cc:343] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:10.513252 622 tablet_service.cc:1511] Processing CreateTablet for tablet 5a892c4bc5694118b514cc1baa56601d (DEFAULT_TABLE table=default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.514420 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a892c4bc5694118b514cc1baa56601d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.519800 855 tablet_service.cc:1511] Processing CreateTablet for tablet 5a892c4bc5694118b514cc1baa56601d (DEFAULT_TABLE table=default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.521080 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a892c4bc5694118b514cc1baa56601d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.525424 627 tablet_service.cc:1511] Processing CreateTablet for tablet 05ef1be8eaa84e6bb5586e68268799a0 (DEFAULT_TABLE table=default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.525887 2408 raft_consensus.cc:493] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:10.527020 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ef1be8eaa84e6bb5586e68268799a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.526467 2408 raft_consensus.cc:515] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:10.528029 864 tablet_service.cc:1511] Processing CreateTablet for tablet 05ef1be8eaa84e6bb5586e68268799a0 (DEFAULT_TABLE table=default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.529211 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ef1be8eaa84e6bb5586e68268799a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.531103 2408 leader_election.cc:290] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:10.555680 635 tablet_service.cc:1511] Processing CreateTablet for tablet 705fc015928140a58db979445068365f (DEFAULT_TABLE table=default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.556938 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705fc015928140a58db979445068365f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.564364 862 tablet_service.cc:1511] Processing CreateTablet for tablet 705fc015928140a58db979445068365f (DEFAULT_TABLE table=default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.565554 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705fc015928140a58db979445068365f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.521792 375 tablet_service.cc:1511] Processing CreateTablet for tablet 5a892c4bc5694118b514cc1baa56601d (DEFAULT_TABLE table=default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.566977 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a892c4bc5694118b514cc1baa56601d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.530004 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:10.535964 2494 consensus_queue.cc:1048] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260810 07:53:10.578990 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]). Expected version 3 got 2
W20260810 07:53:10.536301 309 consensus_peers.cc:597] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0098aaa7be644c9885167dc1ee4929b5. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:10.581840 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 45649.1us
I20260810 07:53:10.582012 377 tablet_service.cc:1511] Processing CreateTablet for tablet 705fc015928140a58db979445068365f (DEFAULT_TABLE table=default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.548698 385 tablet_service.cc:1511] Processing CreateTablet for tablet 05ef1be8eaa84e6bb5586e68268799a0 (DEFAULT_TABLE table=default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.583120 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705fc015928140a58db979445068365f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.583667 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ef1be8eaa84e6bb5586e68268799a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.587504 2606 tablet.cc:1724] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:10.599210 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:10.599400 635 tablet_service.cc:1511] Processing CreateTablet for tablet 433404aa29574d819371fe090e670095 (DEFAULT_TABLE table=default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:10.500669 2591 log.cc:927] Time spent T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.156s user 0.001s sys 0.000s
I20260810 07:53:10.600497 635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 433404aa29574d819371fe090e670095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.605312 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]). Expected version 3 got 0
I20260810 07:53:10.608420 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:10.610016 855 tablet_service.cc:1511] Processing CreateTablet for tablet 433404aa29574d819371fe090e670095 (DEFAULT_TABLE table=default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.611285 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 433404aa29574d819371fe090e670095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.612324 367 tablet_service.cc:1511] Processing CreateTablet for tablet 433404aa29574d819371fe090e670095 (DEFAULT_TABLE table=default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.613354 367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 433404aa29574d819371fe090e670095. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:10.559158 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 77346.6us
W20260810 07:53:10.622548 308 consensus_peers.cc:597] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:10.631345 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:10.632282 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:10.633030 32461 catalog_manager.cc:5719] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:10.649389 856 tablet_service.cc:1511] Processing CreateTablet for tablet a9f5a4321a504ed59e0e2ba21ac801f7 (DEFAULT_TABLE table=default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.649657 619 tablet_service.cc:1511] Processing CreateTablet for tablet a9f5a4321a504ed59e0e2ba21ac801f7 (DEFAULT_TABLE table=default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.650596 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f5a4321a504ed59e0e2ba21ac801f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.652099 1047 tablet_bootstrap.cc:654] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:10.432534 2397 raft_consensus.cc:2272] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:10.654079 624 tablet_service.cc:1511] Processing CreateTablet for tablet 44ff256b314e479d981333d9cbfc1656 (DEFAULT_TABLE table=default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.655171 2397 ts_tablet_manager.cc:1916] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:10.368510 1064 tablet_bootstrap.cc:654] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:10.664113 377 tablet_service.cc:1511] Processing CreateTablet for tablet a9f5a4321a504ed59e0e2ba21ac801f7 (DEFAULT_TABLE table=default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.666465 32464 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70] alter complete (version 3)
W20260810 07:53:10.667523 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef32f7951f8640e9a13b069816b1cdd5 consensus state has no leader
I20260810 07:53:10.668028 32467 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 295) took 3040 ms. Trace:
I20260810 07:53:10.668936 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a04e29d3a49b4bd891be79e0fad6ec51
I20260810 07:53:10.670059 377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f5a4321a504ed59e0e2ba21ac801f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.674090 32468 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 301) took 1280 ms. Trace:
I20260810 07:53:10.678701 862 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:10.668955 32463 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet a9be900b701a45728a44bf26350a26ab
I20260810 07:53:10.680912 385 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.681979 32463 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 287) took 4720 ms. Trace:
I20260810 07:53:10.682262 1071 tablet_bootstrap.cc:492] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:10.682685 1071 ts_tablet_manager.cc:1403] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.771s user 0.011s sys 0.004s
I20260810 07:53:10.682269 32463 rpcz_store.cc:276] 0810 07:53:05.961139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.978830 (+2017691us) service_pool.cc:224] Handling call
0810 07:53:08.318147 (+339317us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:08.389617 (+ 71470us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:08.389746 (+ 129us) catalog_manager.cc:2850] Locking tablets
0810 07:53:08.407412 (+ 17666us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:08.896850 (+489438us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:08.897039 (+ 189us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:10.668856 (+1771817us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:10.681916 (+ 13060us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.437787 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.444094 (+ 6307us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.444122 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:08.444137 (+ 15us) tablet.cc:662] Decoding operations
0810 07:53:08.444521 (+ 384us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.444567 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.444592 (+ 25us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.444925 (+ 333us) tablet.cc:701] Row locks acquired
0810 07:53:08.444955 (+ 30us) write_op.cc:260] PREPARE: finished
0810 07:53:08.445079 (+ 124us) write_op.cc:270] Start()
0810 07:53:08.445185 (+ 106us) write_op.cc:276] Timestamp: P: 1786348388445006 usec, L: 0
0810 07:53:08.445212 (+ 27us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.445837 (+ 625us) log.cc:844] Serialized 731 byte log entry
0810 07:53:08.893598 (+447761us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.893800 (+ 202us) write_op.cc:301] APPLY: starting
0810 07:53:08.893851 (+ 51us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.893999 (+ 148us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.894265 (+ 266us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.894664 (+ 399us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.894847 (+ 183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:08.894919 (+ 72us) write_op.cc:312] APPLY: finished
0810 07:53:08.895207 (+ 288us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.895772 (+ 565us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.895839 (+ 67us) write_op.cc:454] Released schema lock
0810 07:53:08.895870 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":1455,"apply.run_wall_time_us":2241,"num_ops":2,"prepare.queue_time_us":23980,"prepare.run_cpu_time_us":2689,"prepare.run_wall_time_us":20030,"raft.queue_time_us":11507,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":17,"replication_time_us":448301}]]}
I20260810 07:53:10.688587 852 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:10.690882 369 tablet_service.cc:1511] Processing CreateTablet for tablet 44ff256b314e479d981333d9cbfc1656 (DEFAULT_TABLE table=default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.691273 385 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.692227 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ff256b314e479d981333d9cbfc1656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.693531 2459 raft_consensus.cc:515] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.693691 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_b0c0af5d463242c4acede6ca01119e32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:10.668335 32467 rpcz_store.cc:276] 0810 07:53:07.627280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.900469 (+1273189us) service_pool.cc:224] Handling call
0810 07:53:08.912304 (+ 11835us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:08.913693 (+ 1389us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:08.914251 (+ 558us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:08.914453 (+ 202us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:09.558833 (+644380us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:10.667968 (+1109135us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.999478 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.999978 (+ 500us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:09.000014 (+ 36us) write_op.cc:435] Acquired schema lock
0810 07:53:09.000032 (+ 18us) tablet.cc:662] Decoding operations
0810 07:53:09.000347 (+ 315us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:09.000394 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:09.000425 (+ 31us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:09.000604 (+ 179us) tablet.cc:701] Row locks acquired
0810 07:53:09.000624 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:09.003793 (+ 3169us) write_op.cc:270] Start()
0810 07:53:09.003992 (+ 199us) write_op.cc:276] Timestamp: P: 1786348389000668 usec, L: 0
0810 07:53:09.004014 (+ 22us) op_driver.cc:348] REPLICATION: starting
0810 07:53:09.008432 (+ 4418us) log.cc:844] Serialized 405 byte log entry
0810 07:53:09.545171 (+536739us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.545711 (+ 540us) write_op.cc:301] APPLY: starting
0810 07:53:09.545802 (+ 91us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.545965 (+ 163us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.546063 (+ 98us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.546308 (+ 245us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.546506 (+ 198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:09.546571 (+ 65us) write_op.cc:312] APPLY: finished
0810 07:53:09.558173 (+ 11602us) log.cc:844] Serialized 23 byte log entry
0810 07:53:09.558425 (+ 252us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.558519 (+ 94us) write_op.cc:454] Released schema lock
0810 07:53:09.558576 (+ 57us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:09.558948 (+ 372us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.558993 (+ 45us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":156,"apply.run_cpu_time_us":1961,"apply.run_wall_time_us":13259,"mutex_wait_us":69783,"num_ops":1,"prepare.queue_time_us":72817,"prepare.run_cpu_time_us":2344,"prepare.run_wall_time_us":10464,"replication_time_us":540989,"spinlock_wait_cycles":67968}]]}
I20260810 07:53:10.695567 2459 leader_election.cc:290] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:10.696241 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:10.696880 308 consensus_peers.cc:597] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:10.699203 2382 consensus_queue.cc:1048] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:10.702051 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:10.674387 32468 rpcz_store.cc:276] 0810 07:53:09.394089 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.734979 (+340890us) service_pool.cc:224] Handling call
0810 07:53:09.776236 (+ 41257us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:09.777290 (+ 1054us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:09.778096 (+ 806us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:09.778400 (+ 304us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:09.813573 (+ 35173us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:10.674028 (+860455us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:09.796658 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:09.797228 (+ 570us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:09.797266 (+ 38us) write_op.cc:435] Acquired schema lock
0810 07:53:09.797287 (+ 21us) tablet.cc:662] Decoding operations
0810 07:53:09.797624 (+ 337us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:09.797667 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:09.797693 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:09.797872 (+ 179us) tablet.cc:701] Row locks acquired
0810 07:53:09.797892 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:09.798052 (+ 160us) write_op.cc:270] Start()
0810 07:53:09.798158 (+ 106us) write_op.cc:276] Timestamp: P: 1786348389797995 usec, L: 0
0810 07:53:09.798177 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:09.798932 (+ 755us) log.cc:844] Serialized 405 byte log entry
0810 07:53:09.810308 (+ 11376us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.811292 (+ 984us) write_op.cc:301] APPLY: starting
0810 07:53:09.811372 (+ 80us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.811509 (+ 137us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.811625 (+ 116us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.811869 (+ 244us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.812043 (+ 174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:09.812108 (+ 65us) write_op.cc:312] APPLY: finished
0810 07:53:09.812339 (+ 231us) log.cc:844] Serialized 23 byte log entry
0810 07:53:09.812520 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.812608 (+ 88us) write_op.cc:454] Released schema lock
0810 07:53:09.812650 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:09.814016 (+ 1366us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.814055 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":774,"apply.run_cpu_time_us":1672,"apply.run_wall_time_us":2692,"num_ops":1,"prepare.queue_time_us":16541,"prepare.run_cpu_time_us":2576,"prepare.run_wall_time_us":4423,"raft.queue_time_us":95,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":716,"replication_time_us":12004}]]}
I20260810 07:53:10.702620 888 raft_consensus.cc:3060] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:10.703372 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet d301af14299040528cf3a033ff5177f8 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:10.704051 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_feb396bb0c8345699ece4d8e1245375b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:10.705564 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feb396bb0c8345699ece4d8e1245375b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:10.709075 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cec929b5f0e14f6586ed50137ecd741b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:10.695156 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0c0af5d463242c4acede6ca01119e32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:10.720973 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:10.724084 2604 log.cc:927] Time spent T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260810 07:53:10.724948 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:10.725085 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 30) took 3538 ms (client timeout 1000 ms). Trace:
I20260810 07:53:10.726833 32464 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b] alter complete (version 1)
W20260810 07:53:10.728076 32465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 31) took 2187 ms (client timeout 1000 ms). Trace:
W20260810 07:53:10.728410 32465 rpcz_store.cc:269] 0810 07:53:08.540254 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.589933 (+ 49679us) service_pool.cc:224] Handling call
0810 07:53:10.728008 (+2138075us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":33}
I20260810 07:53:10.729894 2412 ts_tablet_manager.cc:1929] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:10.729665 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_aff6b5cbe0e8421da86a454f107e16d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:10.730295 2412 log.cc:1199] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:10.731104 2412 ts_tablet_manager.cc:1950] T c292cb975c504ad8abf03e913140c7a9 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:10.733016 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:10.734470 864 tablet_service.cc:1511] Processing CreateTablet for tablet 44ff256b314e479d981333d9cbfc1656 (DEFAULT_TABLE table=default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:10.725379 32469 rpcz_store.cc:269] 0810 07:53:07.186680 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.426672 (+239992us) service_pool.cc:224] Handling call
0810 07:53:10.725007 (+3298335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:08.458214 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:08.465354 (+ 7140us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:08.465392 (+ 38us) write_op.cc:435] Acquired schema lock
0810 07:53:08.465408 (+ 16us) tablet.cc:662] Decoding operations
0810 07:53:08.465793 (+ 385us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:08.465832 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:08.465858 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:08.466174 (+ 316us) tablet.cc:701] Row locks acquired
0810 07:53:08.466188 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:08.466328 (+ 140us) write_op.cc:270] Start()
0810 07:53:08.466419 (+ 91us) write_op.cc:276] Timestamp: P: 1786348388466247 usec, L: 0
0810 07:53:08.466433 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:08.500437 (+ 34004us) log.cc:844] Serialized 913 byte log entry
0810 07:53:08.896406 (+395969us) op_driver.cc:464] REPLICATION: finished
0810 07:53:08.905049 (+ 8643us) write_op.cc:301] APPLY: starting
0810 07:53:08.905124 (+ 75us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:08.905235 (+ 111us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:08.905334 (+ 99us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:08.905598 (+ 264us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:08.905736 (+ 138us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:08.905777 (+ 41us) write_op.cc:312] APPLY: finished
0810 07:53:08.905952 (+ 175us) log.cc:844] Serialized 29 byte log entry
0810 07:53:08.906595 (+ 643us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.906690 (+ 95us) write_op.cc:454] Released schema lock
0810 07:53:08.906735 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:08.933297 (+ 26562us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:08.933338 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":436413,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":8470,"apply.run_cpu_time_us":1521,"apply.run_wall_time_us":28173,"num_ops":2,"prepare.queue_time_us":37448,"prepare.run_cpu_time_us":2682,"prepare.run_wall_time_us":69458,"raft.queue_time_us":26790,"raft.run_cpu_time_us":16,"raft.run_wall_time_us":17,"replication_time_us":429886}]]}
I20260810 07:53:10.738449 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_fee6b63b40ed407dbcf843cd9b93b336"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:10.740942 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fee6b63b40ed407dbcf843cd9b93b336 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:10.756664 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aff6b5cbe0e8421da86a454f107e16d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:10.757223 574 leader_election.cc:343] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:10.757624 574 leader_election.cc:304] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:10.758339 633 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:10.761003 1064 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:10.761524 1064 ts_tablet_manager.cc:1403] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.593s user 0.007s sys 0.005s
I20260810 07:53:10.774405 1071 raft_consensus.cc:359] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.775393 852 tablet_service.cc:1511] Processing CreateTablet for tablet dd79131856f54395a3c5fb551961d4f7 (DEFAULT_TABLE table=default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.788923 862 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02aece0af740828f4009706445e60c (DEFAULT_TABLE table=default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.794809 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet d301af14299040528cf3a033ff5177f8
I20260810 07:53:10.797231 859 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:10.796541 32468 catalog_manager.cc:6807] ReplaceTablet: tablet d301af14299040528cf3a033ff5177f8 deleted and replaced by tablet 4fb366297938415382cf6555ed052956
I20260810 07:53:10.798892 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet ef32f7951f8640e9a13b069816b1cdd5 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:10.813350 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ff256b314e479d981333d9cbfc1656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.817242 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02aece0af740828f4009706445e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.821270 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd79131856f54395a3c5fb551961d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.830713 855 tablet_service.cc:1511] Processing CreateTablet for tablet 666f36dafa92421cae47c94593ed8899 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:10.840871 32460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 44) took 3869 ms (client timeout 1000 ms). Trace:
I20260810 07:53:10.848479 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666f36dafa92421cae47c94593ed8899. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:10.848879 32460 rpcz_store.cc:269] 0810 07:53:06.971941 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.403413 (+431472us) service_pool.cc:224] Handling call
0810 07:53:10.840795 (+3437382us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:09.855644 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:09.856171 (+ 527us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:09.856203 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:53:09.856222 (+ 19us) tablet.cc:662] Decoding operations
0810 07:53:09.856550 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:09.856599 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:09.856629 (+ 30us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:09.856830 (+ 201us) tablet.cc:701] Row locks acquired
0810 07:53:09.856847 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:53:09.856980 (+ 133us) write_op.cc:270] Start()
0810 07:53:09.857077 (+ 97us) write_op.cc:276] Timestamp: P: 1786348389856925 usec, L: 0
0810 07:53:09.857097 (+ 20us) op_driver.cc:348] REPLICATION: starting
0810 07:53:09.859258 (+ 2161us) log.cc:844] Serialized 533 byte log entry
0810 07:53:09.888859 (+ 29601us) op_driver.cc:464] REPLICATION: finished
0810 07:53:09.910268 (+ 21409us) write_op.cc:301] APPLY: starting
0810 07:53:09.910354 (+ 86us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:09.910460 (+ 106us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:09.910547 (+ 87us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:09.910750 (+ 203us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:09.910878 (+ 128us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:09.910922 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:09.911112 (+ 190us) log.cc:844] Serialized 23 byte log entry
0810 07:53:09.911772 (+ 660us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.911847 (+ 75us) write_op.cc:454] Released schema lock
0810 07:53:09.911883 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:09.912136 (+ 253us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:09.912166 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":156962,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":21204,"apply.run_cpu_time_us":1373,"apply.run_wall_time_us":1849,"num_ops":1,"prepare.queue_time_us":345,"prepare.run_cpu_time_us":2551,"prepare.run_wall_time_us":19780,"raft.queue_time_us":7468,"raft.run_cpu_time_us":786,"raft.run_wall_time_us":785,"replication_time_us":31652}]]}
W20260810 07:53:10.872766 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 91029.6us
I20260810 07:53:10.888846 1064 raft_consensus.cc:359] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.889752 1064 raft_consensus.cc:385] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:10.890177 1064 raft_consensus.cc:740] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:10.869052 380 tablet_service.cc:1511] Processing CreateTablet for tablet 666f36dafa92421cae47c94593ed8899 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.891117 1071 raft_consensus.cc:385] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:10.891387 1071 raft_consensus.cc:740] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:10.878727 367 tablet_service.cc:1558] Processing DeleteTablet for tablet dd22d1029ff249a0a9e05862293261ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:00 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.879621 385 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02aece0af740828f4009706445e60c (DEFAULT_TABLE table=default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.891778 368 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.885397 376 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.892134 1071 consensus_queue.cc:260] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.892619 382 tablet_service.cc:1558] Processing DeleteTablet for tablet 54a3bd8369634e4bbfdf8920454357bc with delete_type TABLET_DATA_DELETED (replaced by tablet 23bd442cbd9542ab8a521ededc6b1e90) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.889360 373 tablet_service.cc:1511] Processing CreateTablet for tablet 461d70a71f2c44228f4220e7f5fb9c64 (DEFAULT_TABLE table=default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.890268 371 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.891799 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666f36dafa92421cae47c94593ed8899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.896880 32465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_64987f4480ff416b8bd9688ac5df15a5" } modify_external_catalogs: true
I20260810 07:53:10.897414 32465 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_64987f4480ff416b8bd9688ac5df15a5" } modify_external_catalogs: true
I20260810 07:53:10.898156 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 274) took 1727 ms. Trace:
I20260810 07:53:10.898627 2412 rpcz_store.cc:276] 0810 07:53:09.171218 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.171366 (+ 148us) service_pool.cc:224] Handling call
0810 07:53:10.023306 (+851940us) ts_tablet_manager.cc:1151] Deleting tablet c292cb975c504ad8abf03e913140c7a9
0810 07:53:10.137233 (+113927us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:10.166362 (+ 29129us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:10.729744 (+563382us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:10.898078 (+168334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25884,"tablet-delete.queue_time_us":851244}
I20260810 07:53:10.899590 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_749f194454fe43a88634da1983fc5584" } new_table_name: "default.Table_944abc71980d460d94439635f3a18725" modify_external_catalogs: true
I20260810 07:53:10.900943 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]) successfully deleted
I20260810 07:53:10.902875 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 275) took 1731 ms. Trace:
I20260810 07:53:10.906064 32467 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_af59b24731ec4cf8a7c32525e4891ee7" } new_table_name: "default.Table_857eaa5798b14f07bad571642674f2e8" modify_external_catalogs: true
I20260810 07:53:10.912256 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02aece0af740828f4009706445e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.914324 645 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:10.915890 373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461d70a71f2c44228f4220e7f5fb9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.918244 375 tablet_service.cc:1511] Processing CreateTablet for tablet dd79131856f54395a3c5fb551961d4f7 (DEFAULT_TABLE table=default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.923738 1064 consensus_queue.cc:260] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.929713 645 tablet_service.cc:1511] Processing CreateTablet for tablet dd79131856f54395a3c5fb551961d4f7 (DEFAULT_TABLE table=default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.930107 2624 raft_consensus.cc:493] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:10.930725 646 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:10.930778 2624 raft_consensus.cc:515] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.932333 639 tablet_service.cc:1511] Processing CreateTablet for tablet 666f36dafa92421cae47c94593ed8899 (DEFAULT_TABLE table=default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:10.935238 630 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02aece0af740828f4009706445e60c (DEFAULT_TABLE table=default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:10.927596 384 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57540: Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:10.948498 369 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:10.936421 2541 raft_consensus.cc:493] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:10.903291 2412 rpcz_store.cc:276] 0810 07:53:09.171587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.171811 (+ 224us) service_pool.cc:224] Handling call
0810 07:53:10.902363 (+1730552us) ts_tablet_manager.cc:1151] Deleting tablet 77fa93c49cb54e5fa03034eda6fc2300
0810 07:53:10.902429 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:10.902801 (+ 372us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1729918}
I20260810 07:53:10.952956 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd79131856f54395a3c5fb551961d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:10.952625 2541 raft_consensus.cc:515] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:10.935313 1071 ts_tablet_manager.cc:1434] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.252s user 0.008s sys 0.000s
W20260810 07:53:10.954227 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 77fa93c49cb54e5fa03034eda6fc2300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77fa93c49cb54e5fa03034eda6fc2300 already in progress: creating tablet
I20260810 07:53:10.955459 2541 leader_election.cc:290] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:10.956032 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 276) took 1157 ms. Trace:
I20260810 07:53:10.956610 2412 rpcz_store.cc:276] 0810 07:53:09.798735 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.798918 (+ 183us) service_pool.cc:224] Handling call
0810 07:53:10.955416 (+1156498us) ts_tablet_manager.cc:1151] Deleting tablet e6800738970d47dd8e7cba68945042aa
0810 07:53:10.955489 (+ 73us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:10.955954 (+ 465us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1155748}
I20260810 07:53:10.928900 2408 raft_consensus.cc:2749] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:10.960095 32460 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c849f5bbe83b49f1a5b41b6e28baacac" } modify_external_catalogs: true
I20260810 07:53:10.967809 32460 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_c849f5bbe83b49f1a5b41b6e28baacac" } modify_external_catalogs: true
I20260810 07:53:10.981601 646 tablet_service.cc:1511] Processing CreateTablet for tablet 461d70a71f2c44228f4220e7f5fb9c64 (DEFAULT_TABLE table=default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:10.987424 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 230214us
I20260810 07:53:10.990448 574 leader_election.cc:304] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:10.958424 856 tablet_service.cc:1511] Processing CreateTablet for tablet 461d70a71f2c44228f4220e7f5fb9c64 (DEFAULT_TABLE table=default.Table_514fab926ff74afcbc734cb171022edf [id=f3de5e5548744d66bfa60af4cfa7b279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:11.009085 1064 ts_tablet_manager.cc:1434] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.238s user 0.008s sys 0.000s
I20260810 07:53:11.010649 888 raft_consensus.cc:2468] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:11.038866 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet e6800738970d47dd8e7cba68945042aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6800738970d47dd8e7cba68945042aa already in progress: creating tablet
I20260810 07:53:11.040436 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:11.041435 893 raft_consensus.cc:2393] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:11.030098 2624 leader_election.cc:290] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:11.054788 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 64422.4us
I20260810 07:53:10.981632 2600 tablet.cc:1724] T fc12205a097c496cba5b777b98f27914 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260810 07:53:11.042120 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:11.044792 669 raft_consensus.cc:3060] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:11.072079 2417 tablet_replica.cc:333] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:11.073707 2417 raft_consensus.cc:2243] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:11.081254 2417 raft_consensus.cc:2272] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.085661 2417 ts_tablet_manager.cc:1916] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:11.031661 619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f5a4321a504ed59e0e2ba21ac801f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.089670 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ff256b314e479d981333d9cbfc1656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.095938 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461d70a71f2c44228f4220e7f5fb9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.101420 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666f36dafa92421cae47c94593ed8899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.105213 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02aece0af740828f4009706445e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.110962 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd79131856f54395a3c5fb551961d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.114696 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461d70a71f2c44228f4220e7f5fb9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:11.125797 1064 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:11.128156 2412 tablet_replica.cc:333] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:11.129832 2412 raft_consensus.cc:2243] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:11.130748 2412 raft_consensus.cc:2272] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.134527 2412 ts_tablet_manager.cc:1916] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:11.145668 32450 catalog_manager.cc:5129] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): alter failed for tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]): Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:11.183341 1071 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:11.187029 309 leader_election.cc:304] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
W20260810 07:53:11.192130 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 138614us
I20260810 07:53:11.194722 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:11.193775 2426 raft_consensus.cc:2804] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:11.209537 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet ef32f7951f8640e9a13b069816b1cdd5
I20260810 07:53:11.211262 32468 catalog_manager.cc:6807] ReplaceTablet: tablet ef32f7951f8640e9a13b069816b1cdd5 deleted and replaced by tablet cca378598d0c45178e497fd212638d77
I20260810 07:53:11.215448 864 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:11.215539 2412 ts_tablet_manager.cc:1929] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 07:53:11.215693 792 leader_election.cc:343] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:11.215962 2412 log.cc:1199] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/a04e29d3a49b4bd891be79e0fad6ec51
I20260810 07:53:11.215866 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]). Expected version 1 got 0
W20260810 07:53:11.225461 32463 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7d6c33f86da645e9bf50e45bc229d0a4 consensus state has no leader
I20260810 07:53:11.223626 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]). Expected version 2 got 0
I20260810 07:53:11.229110 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:11.233603 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:11.237010 384 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:11.249476 1064 tablet_bootstrap.cc:654] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:11.260300 2412 ts_tablet_manager.cc:1950] T a04e29d3a49b4bd891be79e0fad6ec51 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:11.270360 2412 tablet_replica.cc:333] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:11.271536 2412 raft_consensus.cc:2243] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:11.273584 2412 raft_consensus.cc:2272] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.273605 2481 raft_consensus.cc:2749] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:11.282418 2412 ts_tablet_manager.cc:1916] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:11.289491 667 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.818s user 0.000s sys 0.021s
W20260810 07:53:11.296337 667 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.825s user 0.000s sys 0.022s
W20260810 07:53:11.297463 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 104075us
I20260810 07:53:11.292230 2382 consensus_queue.cc:1048] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260810 07:53:11.263938 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]) successfully deleted
W20260810 07:53:11.323137 32451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 59232.1us
I20260810 07:53:11.323254 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]). Expected version 1 got 0
I20260810 07:53:11.323935 32470 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9] alter complete (version 3)
I20260810 07:53:11.278960 2426 raft_consensus.cc:697] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:11.367238 2426 consensus_queue.cc:237] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:11.369509 576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.011s sys 0.003s
W20260810 07:53:11.370052 576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s user 0.011s sys 0.003s
I20260810 07:53:11.370214 2642 tablet.cc:1724] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260810 07:53:11.375358 375 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57540: Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
W20260810 07:53:11.380290 573 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s user 0.003s sys 0.010s
W20260810 07:53:11.380601 573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s user 0.003s sys 0.010s
I20260810 07:53:11.383410 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:11.386864 622 consensus_meta.cc:287] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: Time spent flushing consensus metadata: real 0.842s user 0.007s sys 0.005s
W20260810 07:53:11.392850 32467 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82edc39a864f49f8be60e4b36ce3d59c consensus state has no leader
I20260810 07:53:11.395208 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_2aaec545288b4889811f4c7f91fa7218"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:11.410035 32468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:11.416419 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2aaec545288b4889811f4c7f91fa7218 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:11.417356 2397 ts_tablet_manager.cc:1929] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:11.417968 2397 log.cc:1199] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/37b48c88571040f6be8d7cbd388feb58
I20260810 07:53:11.418845 864 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:11.419004 2397 ts_tablet_manager.cc:1950] T 37b48c88571040f6be8d7cbd388feb58 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:53:11.420048 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7d6c33f86da645e9bf50e45bc229d0a4 consensus state has no leader
I20260810 07:53:11.420248 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 262) took 3830 ms. Trace:
I20260810 07:53:11.429950 2647 tablet.cc:1724] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:11.435127 2494 raft_consensus.cc:493] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:11.435652 2494 raft_consensus.cc:515] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.438551 2494 leader_election.cc:290] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:11.440228 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:11.441020 888 raft_consensus.cc:2468] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:11.442422 309 leader_election.cc:304] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:11.443420 2494 raft_consensus.cc:2804] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260810 07:53:11.444393 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 32) took 1693 ms (client timeout 1000 ms). Trace:
I20260810 07:53:11.444581 2494 raft_consensus.cc:493] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:11.445163 2494 raft_consensus.cc:3060] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:11.415584 32468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:11.458755 1071 tablet_bootstrap.cc:654] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:11.466985 1064 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:11.467518 1064 ts_tablet_manager.cc:1403] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.342s user 0.013s sys 0.003s
I20260810 07:53:11.475137 856 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:11.463943 2619 raft_consensus.cc:493] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:11.485190 2619 raft_consensus.cc:515] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.473695 619 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.473937 622 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.474123 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.489495 2619 leader_election.cc:290] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:11.490944 32461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 45) took 1956 ms (client timeout 1000 ms). Trace:
I20260810 07:53:11.490967 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:11.491793 886 raft_consensus.cc:2468] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:11.492980 375 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:11.493099 574 leader_election.cc:304] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:11.494338 2619 raft_consensus.cc:2804] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:11.494757 2619 raft_consensus.cc:493] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:11.495217 2619 raft_consensus.cc:3060] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:11.457304 1047 tablet_bootstrap.cc:492] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:11.498600 1047 ts_tablet_manager.cc:1403] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 1.081s user 0.007s sys 0.008s
I20260810 07:53:11.502130 1047 raft_consensus.cc:359] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.507153 1064 raft_consensus.cc:359] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.507812 1064 raft_consensus.cc:385] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:11.508055 1064 raft_consensus.cc:740] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:11.507948 32463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_30fefb1a7d9d4121ba5ad66228a53849"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:11.509809 32463 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30fefb1a7d9d4121ba5ad66228a53849 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:11.525516 32455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 51370.3us
I20260810 07:53:11.530735 2632 consensus_queue.cc:1048] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:11.538600 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:11.539026 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:53:11.539610 308 leader_election.cc:343] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:11.539829 2494 raft_consensus.cc:515] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.541375 2382 consensus_queue.cc:1048] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:11.541137 1071 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:11.549173 1071 ts_tablet_manager.cc:1403] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.397s user 0.010s sys 0.006s
I20260810 07:53:11.526026 2397 rpcz_store.cc:276] 0810 07:53:07.589891 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.590027 (+ 136us) service_pool.cc:224] Handling call
0810 07:53:10.430295 (+2840268us) ts_tablet_manager.cc:1151] Deleting tablet 37b48c88571040f6be8d7cbd388feb58
0810 07:53:10.430365 (+ 70us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.302679 (+872314us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:11.417184 (+114505us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:11.420178 (+ 2994us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839644}
I20260810 07:53:11.553787 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 37b48c88571040f6be8d7cbd388feb58 (table default.Table_903e7df2a6cc4ebdb26c728e4c047cfc [id=1117112f21eb4eb9b043be0d2cc58205]) successfully deleted
I20260810 07:53:11.557094 1047 raft_consensus.cc:385] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:11.557399 1047 raft_consensus.cc:740] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:11.558102 1047 consensus_queue.cc:260] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.562148 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:11.563319 32452 consensus_peers.cc:597] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260810 07:53:11.538424 627 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.551600 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:11.444953 32470 rpcz_store.cc:269] 0810 07:53:09.751425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.802973 (+ 51548us) service_pool.cc:224] Handling call
0810 07:53:10.322771 (+519798us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.322805 (+ 34us) write_op.cc:454] Released schema lock
0810 07:53:11.444322 (+1121517us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:10.295012 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:10.295768 (+ 756us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:10.295802 (+ 34us) write_op.cc:435] Acquired schema lock
0810 07:53:10.295824 (+ 22us) tablet.cc:662] Decoding operations
0810 07:53:10.296187 (+ 363us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:10.296240 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:10.296320 (+ 80us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:10.296531 (+ 211us) tablet.cc:701] Row locks acquired
0810 07:53:10.296553 (+ 22us) write_op.cc:260] PREPARE: finished
0810 07:53:10.296710 (+ 157us) write_op.cc:270] Start()
0810 07:53:10.296872 (+ 162us) write_op.cc:276] Timestamp: P: 1786348390296648 usec, L: 0
0810 07:53:10.296890 (+ 18us) op_driver.cc:348] REPLICATION: starting
0810 07:53:10.297873 (+ 983us) log.cc:844] Serialized 533 byte log entry
0810 07:53:10.309415 (+ 11542us) op_driver.cc:464] REPLICATION: finished
0810 07:53:10.309918 (+ 503us) write_op.cc:301] APPLY: starting
0810 07:53:10.310001 (+ 83us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:10.310131 (+ 130us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:10.310279 (+ 148us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:10.310465 (+ 186us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:10.310639 (+ 174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:10.310712 (+ 73us) write_op.cc:312] APPLY: finished
0810 07:53:10.310984 (+ 272us) log.cc:844] Serialized 23 byte log entry
0810 07:53:10.311184 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.311250 (+ 66us) write_op.cc:454] Released schema lock
0810 07:53:10.311298 (+ 48us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:53:10.784972 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:10.785440 (+ 468us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:10.785466 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:10.785478 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:10.785771 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:10.785813 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:10.785839 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:10.785998 (+ 159us) tablet.cc:701] Row locks acquired
0810 07:53:10.786015 (+ 17us) write_op.cc:260] PREPARE: finished
0810 07:53:10.786138 (+ 123us) write_op.cc:270] Start()
0810 07:53:10.786238 (+ 100us) write_op.cc:276] Timestamp: P: 1786348390786082 usec, L: 0
0810 07:53:10.786258 (+ 20us) op_driver.cc:348] REPLICATION: starting
0810 07:53:10.817164 (+ 30906us) log.cc:844] Serialized 356 byte log entry
0810 07:53:10.834424 (+ 17260us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf964951 00007f30cf9f7208 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1 00007f30cfe8a50d 00000000005bd8c4 00000000005bd6f9
0810 07:53:10.851494 (+ 17070us) op_driver.cc:464] REPLICATION: finished
0810 07:53:10.851835 (+ 341us) write_op.cc:301] APPLY: starting
0810 07:53:10.851894 (+ 59us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:10.851976 (+ 82us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:10.851994 (+ 18us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:10.852184 (+ 190us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:10.852312 (+ 128us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:10.852361 (+ 49us) write_op.cc:312] APPLY: finished
0810 07:53:10.852573 (+ 212us) log.cc:844] Serialized 23 byte log entry
0810 07:53:10.852708 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.852788 (+ 80us) write_op.cc:454] Released schema lock
0810 07:53:10.852829 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:10.864771 (+ 11942us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.864798 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3237,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":253,"apply.run_cpu_time_us":1613,"apply.run_wall_time_us":1612,"num_ops":1,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":3377,"prepare.run_wall_time_us":26640,"raft.queue_time_us":171,"raft.run_cpu_time_us":665,"raft.run_wall_time_us":664,"replication_time_us":12422}],["op",{"apply.queue_time_us":177,"apply.run_cpu_time_us":1278,"apply.run_wall_time_us":12885,"num_ops":1,"prepare.queue_time_us":33442,"prepare.run_cpu_time_us":3923,"prepare.run_wall_time_us":57360,"raft.queue_time_us":6863,"raft.run_cpu_time_us":720,"raft.run_wall_time_us":719,"replication_time_us":65129,"spinlock_wait_cycles":69740928}]]}
I20260810 07:53:11.575627 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_87f5c1dc7317431c9068a366c6429c52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:11.577268 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87f5c1dc7317431c9068a366c6429c52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:11.361387 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 33) took 2368 ms (client timeout 1000 ms). Trace:
I20260810 07:53:11.578811 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 263) took 3987 ms. Trace:
I20260810 07:53:11.579243 2397 rpcz_store.cc:276] 0810 07:53:07.591031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.591139 (+ 108us) service_pool.cc:224] Handling call
0810 07:53:11.578252 (+3987113us) ts_tablet_manager.cc:1151] Deleting tablet acc89d15b4d544b7bc8abc12a4c6d3e5
0810 07:53:11.578443 (+ 191us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.578734 (+ 291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3986613}
W20260810 07:53:11.580615 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet acc89d15b4d544b7bc8abc12a4c6d3e5 because the tablet was not found. No further retry: Not found: Tablet not found: acc89d15b4d544b7bc8abc12a4c6d3e5
I20260810 07:53:11.581120 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 264) took 3879 ms. Trace:
I20260810 07:53:11.581724 2621 raft_consensus.cc:493] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:11.581493 2397 rpcz_store.cc:276] 0810 07:53:07.701341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.701482 (+ 141us) service_pool.cc:224] Handling call
0810 07:53:11.580777 (+3879295us) ts_tablet_manager.cc:1151] Deleting tablet 449359c8e9fa4756ba149f3dce6e0ce7
0810 07:53:11.580854 (+ 77us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.581063 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3878687}
W20260810 07:53:11.583030 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 449359c8e9fa4756ba149f3dce6e0ce7 because the tablet was not found. No further retry: Not found: Tablet not found: 449359c8e9fa4756ba149f3dce6e0ce7
I20260810 07:53:11.582614 2621 raft_consensus.cc:515] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.554054 1064 consensus_queue.cc:260] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.585130 2412 ts_tablet_manager.cc:1929] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:11.585498 2412 log.cc:1199] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/a9be900b701a45728a44bf26350a26ab
I20260810 07:53:11.585865 413 raft_consensus.cc:2393] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:11.586293 2412 ts_tablet_manager.cc:1950] T a9be900b701a45728a44bf26350a26ab P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:11.587044 2494 leader_election.cc:290] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:11.589231 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:11.589856 892 raft_consensus.cc:3060] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:11.603995 1071 raft_consensus.cc:359] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.604833 1071 raft_consensus.cc:385] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:11.605234 1071 raft_consensus.cc:740] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:11.606146 1071 consensus_queue.cc:260] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.609998 627 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.617654 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 265) took 3915 ms. Trace:
I20260810 07:53:11.624154 2397 rpcz_store.cc:276] 0810 07:53:07.701738 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.701850 (+ 112us) service_pool.cc:224] Handling call
0810 07:53:11.617114 (+3915264us) ts_tablet_manager.cc:1151] Deleting tablet c1faed468bfa4d1ba5d92c0b9b194781
0810 07:53:11.617217 (+ 103us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.617568 (+ 351us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3914689}
I20260810 07:53:11.626296 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 266) took 3923 ms. Trace:
W20260810 07:53:11.633368 792 leader_election.cc:343] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:11.633814 792 leader_election.cc:304] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:11.634969 2624 raft_consensus.cc:2749] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:11.639930 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet a9be900b701a45728a44bf26350a26ab (table default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]) successfully deleted
W20260810 07:53:11.641654 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 0098aaa7be644c9885167dc1ee4929b5 because the tablet was not found. No further retry: Not found: Tablet not found: 0098aaa7be644c9885167dc1ee4929b5
W20260810 07:53:11.642467 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because the tablet was not found. No further retry: Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035
W20260810 07:53:11.643178 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 45c5e309389d402092f59c801923854a because the tablet was not found. No further retry: Not found: Tablet not found: 45c5e309389d402092f59c801923854a
I20260810 07:53:11.643903 2412 tablet_replica.cc:333] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:11.645206 1047 ts_tablet_manager.cc:1434] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.146s user 0.007s sys 0.000s
W20260810 07:53:11.645463 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 104687us
W20260810 07:53:11.644309 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 3fe822aa30b54fb091fd62ed8b6e22ff because the tablet was not found. No further retry: Not found: Tablet not found: 3fe822aa30b54fb091fd62ed8b6e22ff
I20260810 07:53:11.647682 1047 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:11.648144 627 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.651907 630 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:11.654136 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet c1faed468bfa4d1ba5d92c0b9b194781 because the tablet was not found. No further retry: Not found: Tablet not found: c1faed468bfa4d1ba5d92c0b9b194781
I20260810 07:53:11.648362 2540 consensus_queue.cc:1048] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260810 07:53:11.661039 1064 ts_tablet_manager.cc:1434] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.193s user 0.005s sys 0.005s
W20260810 07:53:11.577960 32464 rpcz_store.cc:269] 0810 07:53:08.993222 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.175835 (+182613us) service_pool.cc:224] Handling call
0810 07:53:10.180856 (+1005021us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.180883 (+ 27us) write_op.cc:454] Released schema lock
0810 07:53:11.361309 (+1180426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:10.157012 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:10.157507 (+ 495us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:10.157535 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:10.157551 (+ 16us) tablet.cc:662] Decoding operations
0810 07:53:10.157853 (+ 302us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:10.157900 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:10.157926 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:10.158140 (+ 214us) tablet.cc:701] Row locks acquired
0810 07:53:10.158154 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:10.158281 (+ 127us) write_op.cc:270] Start()
0810 07:53:10.158393 (+ 112us) write_op.cc:276] Timestamp: P: 1786348390158238 usec, L: 0
0810 07:53:10.158406 (+ 13us) op_driver.cc:348] REPLICATION: starting
0810 07:53:10.158988 (+ 582us) log.cc:844] Serialized 356 byte log entry
0810 07:53:10.171308 (+ 12320us) op_driver.cc:464] REPLICATION: finished
0810 07:53:10.171650 (+ 342us) write_op.cc:301] APPLY: starting
0810 07:53:10.171723 (+ 73us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:10.171846 (+ 123us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:10.171981 (+ 135us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:10.172211 (+ 230us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:10.172381 (+ 170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:10.172440 (+ 59us) write_op.cc:312] APPLY: finished
0810 07:53:10.172687 (+ 247us) log.cc:844] Serialized 23 byte log entry
0810 07:53:10.172884 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.172981 (+ 97us) write_op.cc:454] Released schema lock
0810 07:53:10.173022 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:53:10.708612 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:10.710681 (+ 2069us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:10.710711 (+ 30us) write_op.cc:435] Acquired schema lock
0810 07:53:10.710729 (+ 18us) tablet.cc:662] Decoding operations
0810 07:53:10.711033 (+ 304us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:10.711079 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:10.711108 (+ 29us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:10.711278 (+ 170us) tablet.cc:701] Row locks acquired
0810 07:53:10.711294 (+ 16us) write_op.cc:260] PREPARE: finished
0810 07:53:10.711430 (+ 136us) write_op.cc:270] Start()
0810 07:53:10.711524 (+ 94us) write_op.cc:276] Timestamp: P: 1786348390711377 usec, L: 0
0810 07:53:10.711545 (+ 21us) op_driver.cc:348] REPLICATION: starting
0810 07:53:10.712127 (+ 582us) log.cc:844] Serialized 356 byte log entry
0810 07:53:10.722312 (+ 10185us) op_driver.cc:464] REPLICATION: finished
0810 07:53:10.722750 (+ 438us) write_op.cc:301] APPLY: starting
0810 07:53:10.722804 (+ 54us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:10.722898 (+ 94us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:10.722979 (+ 81us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:10.723226 (+ 247us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:10.723390 (+ 164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:10.723435 (+ 45us) write_op.cc:312] APPLY: finished
0810 07:53:10.723619 (+ 184us) log.cc:844] Serialized 23 byte log entry
0810 07:53:10.723766 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.723829 (+ 63us) write_op.cc:454] Released schema lock
0810 07:53:10.723857 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:10.724102 (+ 245us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.724125 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":189,"apply.run_cpu_time_us":1586,"apply.run_wall_time_us":1585,"num_ops":1,"prepare.queue_time_us":23771,"prepare.run_cpu_time_us":2330,"prepare.run_wall_time_us":2330,"raft.queue_time_us":1505,"raft.run_cpu_time_us":720,"raft.run_wall_time_us":720,"replication_time_us":12765}],["op",{"apply.queue_time_us":277,"apply.run_cpu_time_us":1318,"apply.run_wall_time_us":1318,"num_ops":1,"prepare.queue_time_us":11946,"prepare.run_cpu_time_us":2311,"prepare.run_wall_time_us":6508,"raft.queue_time_us":104,"raft.run_cpu_time_us":682,"raft.run_wall_time_us":683,"replication_time_us":10668}]]}
W20260810 07:53:11.491303 32461 rpcz_store.cc:269] 0810 07:53:09.534640 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.553217 (+ 18577us) service_pool.cc:224] Handling call
0810 07:53:09.737184 (+183967us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:10.763750 (+1026566us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.763775 (+ 25us) write_op.cc:454] Released schema lock
0810 07:53:11.490868 (+727093us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:10.683773 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:10.684422 (+ 649us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:10.684455 (+ 33us) write_op.cc:435] Acquired schema lock
0810 07:53:10.684472 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:10.685287 (+ 815us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:10.685335 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:10.685361 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:10.685537 (+ 176us) tablet.cc:701] Row locks acquired
0810 07:53:10.685549 (+ 12us) write_op.cc:260] PREPARE: finished
0810 07:53:10.685680 (+ 131us) write_op.cc:270] Start()
0810 07:53:10.685771 (+ 91us) write_op.cc:276] Timestamp: P: 1786348390685633 usec, L: 0
0810 07:53:10.685787 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:10.686511 (+ 724us) log.cc:844] Serialized 533 byte log entry
0810 07:53:10.699616 (+ 13105us) op_driver.cc:464] REPLICATION: finished
0810 07:53:10.699998 (+ 382us) write_op.cc:301] APPLY: starting
0810 07:53:10.700057 (+ 59us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:10.700150 (+ 93us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:10.700232 (+ 82us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:10.700410 (+ 178us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:10.700552 (+ 142us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:10.700597 (+ 45us) write_op.cc:312] APPLY: finished
0810 07:53:10.701340 (+ 743us) log.cc:844] Serialized 23 byte log entry
0810 07:53:10.701519 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:10.701585 (+ 66us) write_op.cc:454] Released schema lock
0810 07:53:10.701615 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5954,"reported_tablets":36,"spinlock_wait_cycles":400978176,"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1263,"apply.run_wall_time_us":1787,"num_ops":1,"prepare.queue_time_us":2434,"prepare.run_cpu_time_us":3019,"prepare.run_wall_time_us":4589,"raft.queue_time_us":112,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":672,"replication_time_us":13733}]]}
I20260810 07:53:11.662386 1064 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:11.677333 2621 leader_election.cc:290] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:11.677454 892 raft_consensus.cc:2468] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:11.678433 2412 raft_consensus.cc:2243] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:11.692055 2412 raft_consensus.cc:2272] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.694136 2412 ts_tablet_manager.cc:1916] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:11.694829 1047 tablet_bootstrap.cc:654] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:11.696103 309 leader_election.cc:304] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
W20260810 07:53:11.700506 574 leader_election.cc:343] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: a9be900b701a45728a44bf26350a26ab
I20260810 07:53:11.713222 2621 consensus_queue.cc:1048] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:11.715658 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 45503.7us
I20260810 07:53:11.719928 2669 tablet.cc:1724] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:11.726629 669 raft_consensus.cc:3060] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:11.727046 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:11.730268 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:11.716568 573 leader_election.cc:304] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:11.738936 624 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57706 (request call id 324) took 1085 ms. Trace:
I20260810 07:53:11.739423 624 rpcz_store.cc:276] 0810 07:53:10.653273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.653396 (+ 123us) service_pool.cc:224] Handling call
0810 07:53:11.075595 (+422199us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:11.075720 (+ 125us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:11.364059 (+288339us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:11.738380 (+374321us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:11.738873 (+ 493us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:11.745512 1064 tablet_bootstrap.cc:654] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:11.746028 2621 raft_consensus.cc:2749] T 07fe1a943230432dbe533e7b3671172b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:11.750239 1047 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:11.750666 1047 ts_tablet_manager.cc:1403] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.103s user 0.007s sys 0.007s
I20260810 07:53:11.736461 2619 raft_consensus.cc:515] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.759639 670 raft_consensus.cc:2393] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
W20260810 07:53:11.765659 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 69629.7us
I20260810 07:53:11.774322 2619 leader_election.cc:290] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:11.776887 373 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57540: Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:11.789499 646 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:11.790989 1071 ts_tablet_manager.cc:1434] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.241s user 0.005s sys 0.003s
I20260810 07:53:11.800477 1064 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:11.800997 1064 ts_tablet_manager.cc:1403] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.139s user 0.012s sys 0.000s
I20260810 07:53:11.804955 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_514fab926ff74afcbc734cb171022edf" } new_table_name: "default.Table_f930ee3bb1ab4902aee3db602c1440ba" modify_external_catalogs: true
I20260810 07:53:11.818408 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9be900b701a45728a44bf26350a26ab" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:11.821429 1071 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:11.823783 32464 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_29e9be4c452b438e82564903ce411e53" } new_table_name: "default.Table_ec23fc68529e443b9b2141c44a5140e9" modify_external_catalogs: true
I20260810 07:53:11.825706 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 2a8f390b0db64e1d96c3d8686274cbbf from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:11.827469 1047 raft_consensus.cc:359] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.828217 1047 raft_consensus.cc:385] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:11.828480 1047 raft_consensus.cc:740] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:11.829192 1047 consensus_queue.cc:260] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.830889 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:53:11.832628 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 43551.4us
W20260810 07:53:11.833837 574 leader_election.cc:343] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:53:11.809165 2638 raft_consensus.cc:2804] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:11.837010 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d301af14299040528cf3a033ff5177f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:11.626874 2397 rpcz_store.cc:276] 0810 07:53:07.702858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.702967 (+ 109us) service_pool.cc:224] Handling call
0810 07:53:11.625811 (+3922844us) ts_tablet_manager.cc:1151] Deleting tablet 2c171d3358e44ae2962ab3c4c377b0f1
0810 07:53:11.625927 (+ 116us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.626228 (+ 301us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3922270}
I20260810 07:53:11.840477 2638 raft_consensus.cc:697] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:11.842192 2638 consensus_queue.cc:237] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.854239 1047 ts_tablet_manager.cc:1434] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.103s user 0.008s sys 0.000s
I20260810 07:53:11.863305 2412 ts_tablet_manager.cc:1929] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:11.872197 2412 log.cc:1199] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/d301af14299040528cf3a033ff5177f8
I20260810 07:53:11.874697 1047 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:11.870064 1064 raft_consensus.cc:359] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.875778 1064 raft_consensus.cc:385] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:11.876165 1064 raft_consensus.cc:740] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:11.877089 1064 consensus_queue.cc:260] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.882738 2666 raft_consensus.cc:493] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:11.883292 2666 raft_consensus.cc:515] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.888212 2666 leader_election.cc:290] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:11.890307 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 2c171d3358e44ae2962ab3c4c377b0f1 because the tablet was not found. No further retry: Not found: Tablet not found: 2c171d3358e44ae2962ab3c4c377b0f1
I20260810 07:53:11.894569 1064 ts_tablet_manager.cc:1434] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.093s user 0.002s sys 0.004s
I20260810 07:53:11.896203 1064 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:11.875854 2412 ts_tablet_manager.cc:1950] T d301af14299040528cf3a033ff5177f8 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:11.899364 2412 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 292) took 1102 ms. Trace:
I20260810 07:53:11.906220 2397 tablet_replica.cc:333] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:11.907156 2412 rpcz_store.cc:276] 0810 07:53:10.796948 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.797092 (+ 144us) service_pool.cc:224] Handling call
0810 07:53:11.643749 (+846657us) ts_tablet_manager.cc:1151] Deleting tablet d301af14299040528cf3a033ff5177f8
0810 07:53:11.643785 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.680469 (+ 36684us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b5400085cd4. stack: 00007f49ea319cf3 00007f49ea319bf0 00007f49e9ec7574 0000000000516de4 0000000000516d79 00007f49f141a951 00007f49f14b9742 00007f49f18f9d2c 00007f49f256141a 00007f49f25920e0 00007f49f257864d 00007f49f25785f9 00007f49f2578589 00007f49f2578551 00007f49f257777d 00000000005bd8c4
0810 07:53:11.756515 (+ 76046us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:11.863171 (+106656us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:11.899305 (+ 36134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19845,"spinlock_wait_cycles":31376768,"tablet-delete.queue_time_us":846055}
I20260810 07:53:11.913249 2412 tablet_replica.cc:333] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:11.914834 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet d301af14299040528cf3a033ff5177f8 (table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]) successfully deleted
I20260810 07:53:11.920218 1064 tablet_bootstrap.cc:654] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:11.925354 2397 raft_consensus.cc:2243] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:11.925971 2397 raft_consensus.cc:2272] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.929376 2412 raft_consensus.cc:2243] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:11.929979 2412 raft_consensus.cc:2272] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:11.933779 2412 ts_tablet_manager.cc:1916] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:11.934454 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 111773us
I20260810 07:53:11.975355 1064 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:11.975898 1064 ts_tablet_manager.cc:1403] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.080s user 0.011s sys 0.003s
I20260810 07:53:11.990103 1064 raft_consensus.cc:359] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:11.991011 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_ed081fcbe2354d06836e6fd3ee83a434"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:11.993438 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed081fcbe2354d06836e6fd3ee83a434 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:11.998068 32452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260810 07:53:12.007535 1064 raft_consensus.cc:385] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.008005 1064 raft_consensus.cc:740] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.008951 1064 consensus_queue.cc:260] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.015234 1064 ts_tablet_manager.cc:1434] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260810 07:53:12.020262 573 leader_election.cc:304] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:12.028620 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:12.033679 792 leader_election.cc:343] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.041795 1071 tablet_bootstrap.cc:654] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.042104 2412 ts_tablet_manager.cc:1929] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:53:12.042508 2412 log.cc:1199] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/ef32f7951f8640e9a13b069816b1cdd5
I20260810 07:53:12.012993 2397 ts_tablet_manager.cc:1916] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:12.043290 2412 ts_tablet_manager.cc:1950] T ef32f7951f8640e9a13b069816b1cdd5 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:12.043864 1064 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.046181 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]) successfully deleted
I20260810 07:53:12.047221 2412 tablet_replica.cc:333] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:12.048190 2412 raft_consensus.cc:2243] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:12.048761 2412 raft_consensus.cc:2272] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:12.051381 2412 ts_tablet_manager.cc:1916] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:12.051995 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 235435us
I20260810 07:53:12.032272 670 raft_consensus.cc:3060] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:12.066987 32464 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9b4339a26ad540f4b385f69fd0dcdc5d consensus state has no leader
I20260810 07:53:12.015321 2632 consensus_queue.cc:1048] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260810 07:53:12.075276 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_18cea3a247ec441fa708a7895fb841bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:12.075817 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:12.076301 309 consensus_peers.cc:597] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9be900b701a45728a44bf26350a26ab. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:12.077442 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18cea3a247ec441fa708a7895fb841bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:12.077844 792 leader_election.cc:343] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.078269 792 leader_election.cc:304] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:12.079001 1064 tablet_bootstrap.cc:654] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.080120 2666 raft_consensus.cc:2749] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:12.051403 2382 consensus_queue.cc:1048] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260810 07:53:12.083472 856 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb34e5b988b497c8f72a6c7266c0f2d (DEFAULT_TABLE table=default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.084551 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb34e5b988b497c8f72a6c7266c0f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.091843 373 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb34e5b988b497c8f72a6c7266c0f2d (DEFAULT_TABLE table=default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.098841 864 tablet_service.cc:1511] Processing CreateTablet for tablet 0e15de89beda46f29a650f93f01d4fda (DEFAULT_TABLE table=default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.100018 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e15de89beda46f29a650f93f01d4fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.103312 1064 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.103827 1064 ts_tablet_manager.cc:1403] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.000s
I20260810 07:53:12.106707 1064 raft_consensus.cc:359] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.035681 2417 ts_tablet_manager.cc:1929] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:12.109232 1064 raft_consensus.cc:385] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.109634 1064 raft_consensus.cc:740] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.113610 1047 tablet_bootstrap.cc:654] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.113348 1064 consensus_queue.cc:260] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.116366 1064 ts_tablet_manager.cc:1434] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
W20260810 07:53:12.118119 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 98396.9us
I20260810 07:53:12.119680 573 leader_election.cc:304] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:12.121054 373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb34e5b988b497c8f72a6c7266c0f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.122606 2619 raft_consensus.cc:2749] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:12.123251 2632 raft_consensus.cc:493] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:12.117362 1064 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.129761 645 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb34e5b988b497c8f72a6c7266c0f2d (DEFAULT_TABLE table=default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.131477 646 tablet_service.cc:1511] Processing CreateTablet for tablet 0e15de89beda46f29a650f93f01d4fda (DEFAULT_TABLE table=default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.115805 2417 log.cc:1199] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/a04e29d3a49b4bd891be79e0fad6ec51
I20260810 07:53:12.133631 375 tablet_service.cc:1511] Processing CreateTablet for tablet 0e15de89beda46f29a650f93f01d4fda (DEFAULT_TABLE table=default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.134124 2417 ts_tablet_manager.cc:1950] T a04e29d3a49b4bd891be79e0fad6ec51 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:12.134706 375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e15de89beda46f29a650f93f01d4fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.138185 855 tablet_service.cc:1511] Processing CreateTablet for tablet bf6663cbe7fd43a79265d87c2cc11a62 (DEFAULT_TABLE table=default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.139199 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6663cbe7fd43a79265d87c2cc11a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.140400 624 tablet_service.cc:1511] Processing CreateTablet for tablet bf6663cbe7fd43a79265d87c2cc11a62 (DEFAULT_TABLE table=default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.152153 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb34e5b988b497c8f72a6c7266c0f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.157337 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e15de89beda46f29a650f93f01d4fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.168205 1064 tablet_bootstrap.cc:654] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.168411 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6663cbe7fd43a79265d87c2cc11a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.167246 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 325) took 1410 ms. Trace:
I20260810 07:53:12.186813 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111),
I20260810 07:53:12.187345 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:12.181227 2417 rpcz_store.cc:276] 0810 07:53:10.756701 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.758208 (+ 1507us) service_pool.cc:224] Handling call
0810 07:53:10.839099 (+ 80891us) ts_tablet_manager.cc:1151] Deleting tablet a04e29d3a49b4bd891be79e0fad6ec51
0810 07:53:11.071889 (+232790us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:11.679796 (+607907us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:12.035265 (+355469us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:12.167142 (+131877us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6165,"tablet-delete.queue_time_us":114}
I20260810 07:53:12.190004 2417 tablet_replica.cc:333] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:12.201980 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]) successfully deleted
I20260810 07:53:12.202565 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:12.123749 2632 raft_consensus.cc:515] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.203537 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]). Expected version 1 got 0
I20260810 07:53:12.204694 2632 leader_election.cc:290] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:12.206756 309 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:12.210700 852 tablet_service.cc:1511] Processing CreateTablet for tablet b2cc9ff5b3344dc39e2e560d63f0ce92 (DEFAULT_TABLE table=default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.211107 639 tablet_service.cc:1511] Processing CreateTablet for tablet b2cc9ff5b3344dc39e2e560d63f0ce92 (DEFAULT_TABLE table=default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.211808 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cc9ff5b3344dc39e2e560d63f0ce92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.212128 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cc9ff5b3344dc39e2e560d63f0ce92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.216913 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_f1136e7481f74aaba7d9c2e77b1876f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:12.218709 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1136e7481f74aaba7d9c2e77b1876f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:12.232649 2624 raft_consensus.cc:493] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.233145 2624 raft_consensus.cc:515] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.235077 2624 leader_election.cc:290] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:12.238571 384 tablet_service.cc:1511] Processing CreateTablet for tablet b2cc9ff5b3344dc39e2e560d63f0ce92 (DEFAULT_TABLE table=default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.238953 380 tablet_service.cc:1511] Processing CreateTablet for tablet bf6663cbe7fd43a79265d87c2cc11a62 (DEFAULT_TABLE table=default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.239773 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cc9ff5b3344dc39e2e560d63f0ce92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.242537 1064 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.242993 1064 ts_tablet_manager.cc:1403] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.126s user 0.010s sys 0.002s
I20260810 07:53:12.261718 1064 raft_consensus.cc:359] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.262377 1064 raft_consensus.cc:385] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.262662 1064 raft_consensus.cc:740] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.263808 855 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb366297938415382cf6555ed052956 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.265111 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb366297938415382cf6555ed052956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.204526 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:12.267632 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6663cbe7fd43a79265d87c2cc11a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.266875 2412 ts_tablet_manager.cc:1929] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:12.276747 864 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5a69fed75047f18522bfee894c769b (DEFAULT_TABLE table=default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.278534 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5a69fed75047f18522bfee894c769b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.279160 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:12.280488 792 leader_election.cc:343] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.280623 385 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb366297938415382cf6555ed052956 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.281869 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb366297938415382cf6555ed052956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.284904 2412 log.cc:1199] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:12.285636 2412 ts_tablet_manager.cc:1950] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:12.288261 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 2d5ab03facf8477eada5d1ac3ec7f5ef (table default.Table_ae927dcca7ec42b99eee31143de3bf0b [id=b70f05a8651b408393dcb8a5cae4507d]) successfully deleted
I20260810 07:53:12.289240 1064 consensus_queue.cc:260] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.290349 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 2a8f390b0db64e1d96c3d8686274cbbf
I20260810 07:53:12.291780 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 2a8f390b0db64e1d96c3d8686274cbbf deleted and replaced by tablet 87ae23cb445e43fc9dc2f45097190fec
I20260810 07:53:12.314008 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:12.316150 862 tablet_service.cc:1511] Processing CreateTablet for tablet e20d46d62cdc48669d7474fae00660d7 (DEFAULT_TABLE table=default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.317219 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d46d62cdc48669d7474fae00660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.321768 369 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf with delete_type TABLET_DATA_DELETED (replaced by tablet 87ae23cb445e43fc9dc2f45097190fec) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:12.322082 1064 ts_tablet_manager.cc:1434] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.079s user 0.006s sys 0.000s
I20260810 07:53:12.323431 1064 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.275825 32469 catalog_manager.cc:5719] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:12.331787 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:12.332508 32469 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]). Expected version 1 got 0
I20260810 07:53:12.333086 371 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5a69fed75047f18522bfee894c769b (DEFAULT_TABLE table=default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.334564 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5a69fed75047f18522bfee894c769b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.349979 2624 raft_consensus.cc:493] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.350500 2624 raft_consensus.cc:515] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.350996 369 tablet_service.cc:1511] Processing CreateTablet for tablet e20d46d62cdc48669d7474fae00660d7 (DEFAULT_TABLE table=default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:12.351490 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet c292cb975c504ad8abf03e913140c7a9 because the tablet was not found. No further retry: Not found: Tablet not found: c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:12.352056 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d46d62cdc48669d7474fae00660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.352479 2624 leader_election.cc:290] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:12.367177 856 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf with delete_type TABLET_DATA_DELETED (replaced by tablet 87ae23cb445e43fc9dc2f45097190fec) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:12.371826 32451 catalog_manager.cc:5030] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 2a8f390b0db64e1d96c3d8686274cbbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a8f390b0db64e1d96c3d8686274cbbf already in progress: creating tablet
I20260810 07:53:12.380191 1064 tablet_bootstrap.cc:654] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.398452 630 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb366297938415382cf6555ed052956 (DEFAULT_TABLE table=default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.402833 627 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5a69fed75047f18522bfee894c769b (DEFAULT_TABLE table=default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.404074 635 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8f390b0db64e1d96c3d8686274cbbf with delete_type TABLET_DATA_DELETED (replaced by tablet 87ae23cb445e43fc9dc2f45097190fec) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:12.405795 622 tablet_service.cc:1511] Processing CreateTablet for tablet e20d46d62cdc48669d7474fae00660d7 (DEFAULT_TABLE table=default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:12.416913 1064 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.417338 1064 ts_tablet_manager.cc:1403] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.094s user 0.011s sys 0.000s
I20260810 07:53:12.419996 1064 raft_consensus.cc:359] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.426643 1064 raft_consensus.cc:385] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.426960 1064 raft_consensus.cc:740] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.429008 1047 tablet_bootstrap.cc:492] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:12.427727 1064 consensus_queue.cc:260] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.453140 1064 ts_tablet_manager.cc:1434] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.035s user 0.004s sys 0.003s
I20260810 07:53:12.454385 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb366297938415382cf6555ed052956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.465094 1064 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.482643 2663 raft_consensus.cc:493] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.483276 2663 raft_consensus.cc:515] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.488565 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:12.489319 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9b4339a26ad540f4b385f69fd0dcdc5d consensus state has no leader
I20260810 07:53:12.489388 888 raft_consensus.cc:2468] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:12.490401 1064 tablet_bootstrap.cc:654] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.490648 2624 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.491086 2624 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.491370 574 leader_election.cc:304] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:12.492238 2616 raft_consensus.cc:2804] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:12.492587 2616 raft_consensus.cc:493] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:12.492941 2616 raft_consensus.cc:3060] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.493122 2624 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:12.497354 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20d46d62cdc48669d7474fae00660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.504065 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:12.506498 627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5a69fed75047f18522bfee894c769b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:12.506682 1064 tablet_bootstrap.cc:492] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.507148 1064 ts_tablet_manager.cc:1403] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.002s
I20260810 07:53:12.510362 1064 raft_consensus.cc:359] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.511135 1064 raft_consensus.cc:385] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.511447 1064 raft_consensus.cc:740] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:12.513187 792 leader_election.cc:343] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.513589 792 leader_election.cc:304] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:12.513932 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:12.514504 677 raft_consensus.cc:2468] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
W20260810 07:53:12.514694 792 leader_election.cc:343] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.515357 2624 raft_consensus.cc:2749] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:12.521432 792 leader_election.cc:304] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:12.522233 2624 raft_consensus.cc:2804] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:12.522562 2624 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:12.522874 2624 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.525113 1064 consensus_queue.cc:260] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.526456 1047 ts_tablet_manager.cc:1403] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.652s user 0.015s sys 0.002s
W20260810 07:53:12.530346 308 leader_election.cc:343] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:12.530898 308 leader_election.cc:304] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:12.537719 2624 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.548178 2624 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:12.549520 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:12.550266 677 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.557521 1064 ts_tablet_manager.cc:1434] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.050s user 0.006s sys 0.001s
I20260810 07:53:12.560982 2426 raft_consensus.cc:2749] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:12.567873 1064 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.568782 2663 leader_election.cc:290] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:12.573655 2616 raft_consensus.cc:515] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.578846 1064 tablet_bootstrap.cc:654] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.543092 1047 raft_consensus.cc:359] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:12.607714 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 77395.3us
I20260810 07:53:12.619320 2426 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.619776 2426 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.621837 2426 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:12.622934 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 48035.8us
I20260810 07:53:12.626356 1064 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.626884 1064 ts_tablet_manager.cc:1403] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.059s user 0.011s sys 0.000s
I20260810 07:53:12.629520 1064 raft_consensus.cc:359] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.630213 1064 raft_consensus.cc:385] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.630478 1064 raft_consensus.cc:740] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.631129 1064 consensus_queue.cc:260] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.593314 1047 raft_consensus.cc:385] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.640383 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:12.643287 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:12.645047 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:12.646752 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:12.647285 893 raft_consensus.cc:2468] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:12.656147 1064 ts_tablet_manager.cc:1434] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.029s user 0.007s sys 0.001s
W20260810 07:53:12.663478 792 leader_election.cc:343] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.663853 792 leader_election.cc:304] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:12.664487 2624 raft_consensus.cc:2749] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:12.665488 792 leader_election.cc:343] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:12.666245 792 leader_election.cc:343] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:12.670346 2713 consensus_queue.cc:1048] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:12.674101 1064 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.636034 1047 raft_consensus.cc:740] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.681418 1047 consensus_queue.cc:260] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.681913 309 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:12.683079 1047 ts_tablet_manager.cc:1434] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.156s user 0.004s sys 0.002s
I20260810 07:53:12.688894 2713 raft_consensus.cc:2804] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:12.689333 2713 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:12.689761 2713 raft_consensus.cc:3060] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.690070 2616 leader_election.cc:290] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:12.698293 1071 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:12.698889 1071 ts_tablet_manager.cc:1403] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.878s user 0.013s sys 0.011s
I20260810 07:53:12.703661 1064 tablet_bootstrap.cc:654] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.705329 665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:12.706169 665 raft_consensus.cc:2468] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:12.707551 666 raft_consensus.cc:3060] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.708990 2713 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.717564 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:12.718143 893 raft_consensus.cc:3060] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.725339 893 raft_consensus.cc:2468] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:12.726449 309 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:12.727732 2713 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:12.736750 677 raft_consensus.cc:2468] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:12.737227 1064 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.737775 1064 ts_tablet_manager.cc:1403] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.001s
I20260810 07:53:12.739490 1071 raft_consensus.cc:359] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.740151 1071 raft_consensus.cc:385] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.740415 1071 raft_consensus.cc:740] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.740818 1064 raft_consensus.cc:359] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.741029 1071 consensus_queue.cc:260] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.741638 1064 raft_consensus.cc:385] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.741928 1064 raft_consensus.cc:740] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.743109 1071 ts_tablet_manager.cc:1434] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
I20260810 07:53:12.743986 2459 raft_consensus.cc:2804] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:12.744449 2459 raft_consensus.cc:697] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:12.745563 2459 consensus_queue.cc:237] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.749058 1064 consensus_queue.cc:260] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.754664 1064 ts_tablet_manager.cc:1434] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260810 07:53:12.756937 1064 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:12.774830 667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s user 0.002s sys 0.009s
W20260810 07:53:12.775421 667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s user 0.002s sys 0.009s
I20260810 07:53:12.781757 1064 tablet_bootstrap.cc:654] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.796229 32470 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_4c5928f06794426aa64b4c1a6111d2bf" } new_table_name: "default.Table_843816de8aff46309a751a007da50fa6" modify_external_catalogs: true
I20260810 07:53:12.800067 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_e822693d75fc4bfdad86a06efc56e19c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:12.801568 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e822693d75fc4bfdad86a06efc56e19c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:12.808640 1064 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.809180 1064 ts_tablet_manager.cc:1403] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.002s
I20260810 07:53:12.812915 1064 raft_consensus.cc:359] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.813792 1064 raft_consensus.cc:385] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.814131 1064 raft_consensus.cc:740] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:12.815029 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 148265us
I20260810 07:53:12.814932 1064 consensus_queue.cc:260] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.802306 2621 raft_consensus.cc:2749] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:12.829394 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e0a84839427099a9da06e2b970ce" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:12.830269 677 raft_consensus.cc:3060] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.849110 2417 raft_consensus.cc:2243] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:12.857355 1064 ts_tablet_manager.cc:1434] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.048s user 0.002s sys 0.004s
I20260810 07:53:12.859534 792 leader_election.cc:304] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:12.860423 2624 raft_consensus.cc:2804] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:12.860929 2624 raft_consensus.cc:697] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:12.861744 2624 consensus_queue.cc:237] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.866080 1064 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.866434 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:12.867061 893 raft_consensus.cc:3060] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:12.869215 2777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:12.869711 2777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:12.824875 1047 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:12.880067 893 raft_consensus.cc:2468] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:12.887730 32467 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]). Expected version 1 got 0
I20260810 07:53:12.890969 1064 tablet_bootstrap.cc:654] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:12.891003 32467 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:12.894917 32467 catalog_manager.cc:5719] T f0606ccac5d04aa89d804cf9168018b8 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:12.898876 32467 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
W20260810 07:53:12.902501 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 86520.9us
I20260810 07:53:12.899668 32467 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:12.905797 2616 raft_consensus.cc:493] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:12.904211 32467 catalog_manager.cc:5719] T 07fe1a943230432dbe533e7b3671172b P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:12.906255 2616 raft_consensus.cc:515] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:12.906976 1064 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:12.907464 1064 ts_tablet_manager.cc:1403] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.042s user 0.008s sys 0.002s
I20260810 07:53:12.911193 2417 raft_consensus.cc:2272] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:12.911099 1064 raft_consensus.cc:359] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.911823 1064 raft_consensus.cc:385] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:12.912113 1064 raft_consensus.cc:740] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:12.912849 1064 consensus_queue.cc:260] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:12.914391 2417 ts_tablet_manager.cc:1916] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:12.914572 1064 ts_tablet_manager.cc:1434] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260810 07:53:12.933641 32461 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 461d70a71f2c44228f4220e7f5fb9c64 consensus state has no leader
I20260810 07:53:12.933908 1064 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:12.934245 32461 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 320) took 1504 ms. Trace:
I20260810 07:53:12.934581 32461 rpcz_store.cc:276] 0810 07:53:11.429956 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.672707 (+242751us) service_pool.cc:224] Handling call
0810 07:53:11.804659 (+131952us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c846 00007f30cf429634 00007f30cf4295c2 00007f30cf4294f1 00007f30cf42946c 00007f30cf428722 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:11.805725 (+ 1066us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:11.807284 (+ 1559us) catalog_manager.cc:3751] Apply alter schema
0810 07:53:11.807961 (+ 677us) catalog_manager.cc:3783] Acquired catalog manager lock
0810 07:53:11.808197 (+ 236us) catalog_manager.cc:3941] Updating metadata on disk
0810 07:53:12.071687 (+263490us) catalog_manager.cc:3972] Committing alterations to in-memory state
0810 07:53:12.934177 (+862490us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:11.967435 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:11.970250 (+ 2815us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:11.970403 (+ 153us) write_op.cc:435] Acquired schema lock
0810 07:53:11.970545 (+ 142us) tablet.cc:662] Decoding operations
0810 07:53:11.972161 (+ 1616us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:11.972324 (+ 163us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:11.972594 (+ 270us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:11.973345 (+ 751us) tablet.cc:701] Row locks acquired
0810 07:53:11.973360 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:11.973926 (+ 566us) write_op.cc:270] Start()
0810 07:53:11.974366 (+ 440us) write_op.cc:276] Timestamp: P: 1786348391973730 usec, L: 0
0810 07:53:11.974465 (+ 99us) op_driver.cc:348] REPLICATION: starting
0810 07:53:12.057304 (+ 82839us) log.cc:844] Serialized 405 byte log entry
0810 07:53:12.064529 (+ 7225us) op_driver.cc:464] REPLICATION: finished
0810 07:53:12.069348 (+ 4819us) write_op.cc:301] APPLY: starting
0810 07:53:12.069397 (+ 49us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:12.069484 (+ 87us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:12.069673 (+ 189us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:12.069866 (+ 193us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:12.069999 (+ 133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:12.070042 (+ 43us) write_op.cc:312] APPLY: finished
0810 07:53:12.070210 (+ 168us) log.cc:844] Serialized 23 byte log entry
0810 07:53:12.071413 (+ 1203us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:12.071470 (+ 57us) write_op.cc:454] Released schema lock
0810 07:53:12.071507 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:12.135050 (+ 63543us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf964951 00007f30cf9f7208 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1 00007f30cfe8a50d 00000000005bd8c4 00000000005bd6f9
0810 07:53:12.135792 (+ 742us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:12.135816 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":300338560,"child_traces":[["op",{"apply.queue_time_us":692,"apply.run_cpu_time_us":1215,"apply.run_wall_time_us":2603,"num_ops":1,"prepare.queue_time_us":26088,"prepare.run_cpu_time_us":4349,"prepare.run_wall_time_us":168568,"raft.queue_time_us":580,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":255,"replication_time_us":89933,"spinlock_wait_cycles":107463808}]]}
I20260810 07:53:12.943274 2616 leader_election.cc:290] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:12.945643 575 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 70632.7us
W20260810 07:53:12.950260 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 90793.1us
I20260810 07:53:12.951817 1071 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:12.958029 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:12.962107 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 68525.1us
I20260810 07:53:12.972031 2713 consensus_queue.cc:1048] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:12.968900 2417 ts_tablet_manager.cc:1929] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:12.982373 2397 ts_tablet_manager.cc:1929] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:12.982839 2397 log.cc:1199] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/213fa4838fe847de87091cfba97eaedc
I20260810 07:53:12.983691 2397 ts_tablet_manager.cc:1950] T 213fa4838fe847de87091cfba97eaedc P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:12.984630 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 267) took 5191 ms. Trace:
I20260810 07:53:12.988804 2397 rpcz_store.cc:276] 0810 07:53:07.793108 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:07.793289 (+ 181us) service_pool.cc:224] Handling call
0810 07:53:11.906004 (+4112715us) ts_tablet_manager.cc:1151] Deleting tablet 213fa4838fe847de87091cfba97eaedc
0810 07:53:11.906078 (+ 74us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:12.234029 (+327951us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:12.982191 (+748162us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:12.984542 (+ 2351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8364,"tablet-delete.queue_time_us":4112077}
I20260810 07:53:12.977556 2417 log.cc:1199] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/a9be900b701a45728a44bf26350a26ab
I20260810 07:53:13.003866 2417 ts_tablet_manager.cc:1950] T a9be900b701a45728a44bf26350a26ab P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:12.993496 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:53:13.001833 380 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57540: Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:12.953754 1064 tablet_bootstrap.cc:654] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:13.018751 2624 raft_consensus.cc:493] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.018821 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 327) took 2178 ms. Trace:
I20260810 07:53:13.019177 2417 rpcz_store.cc:276] 0810 07:53:10.840083 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.840207 (+ 124us) service_pool.cc:224] Handling call
0810 07:53:12.189741 (+1349534us) ts_tablet_manager.cc:1151] Deleting tablet a9be900b701a45728a44bf26350a26ab
0810 07:53:12.189833 (+ 92us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:12.922903 (+733070us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:12.968679 (+ 45776us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.018724 (+ 50045us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":713638,"tablet-delete.queue_time_us":1271767}
I20260810 07:53:13.021659 2397 tablet_replica.cc:333] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:13.023476 574 leader_election.cc:304] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:13.024097 32450 catalog_manager.cc:5129] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): alter failed for tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]): Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:13.024394 2616 raft_consensus.cc:2804] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:13.024953 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 213fa4838fe847de87091cfba97eaedc (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]) successfully deleted
I20260810 07:53:13.026520 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:13.027941 2397 raft_consensus.cc:2243] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:13.029724 2417 tablet_replica.cc:333] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:13.030615 2417 raft_consensus.cc:2243] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:13.035305 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 392 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:13.037142 2397 raft_consensus.cc:2272] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.039294 2397 ts_tablet_manager.cc:1916] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.046093 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet a9be900b701a45728a44bf26350a26ab (table default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]) successfully deleted
I20260810 07:53:13.049350 2417 raft_consensus.cc:2272] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.049118 2777 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:13.050339 2624 raft_consensus.cc:515] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.037740 2616 raft_consensus.cc:697] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:13.051939 2616 consensus_queue.cc:237] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.060895 2459 raft_consensus.cc:493] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:13.061507 2459 raft_consensus.cc:515] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.063621 2459 leader_election.cc:290] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:13.067075 309 leader_election.cc:343] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:13.041337 1064 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:13.124835 1064 ts_tablet_manager.cc:1403] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.191s user 0.010s sys 0.008s
I20260810 07:53:13.127619 1064 raft_consensus.cc:359] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.128325 1064 raft_consensus.cc:385] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:13.128585 1064 raft_consensus.cc:740] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:13.129737 1064 consensus_queue.cc:260] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.035332 2783 raft_consensus.cc:493] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.093969 2622 raft_consensus.cc:493] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.134495 2622 raft_consensus.cc:515] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.136446 2622 leader_election.cc:290] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:13.137058 2622 raft_consensus.cc:493] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.137516 2622 raft_consensus.cc:515] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.143893 2622 leader_election.cc:290] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:13.144790 2397 ts_tablet_manager.cc:1929] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:13.145313 2397 log.cc:1199] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/7000d25f3bfe4c99be54a0f4789cc96c
I20260810 07:53:13.146378 2397 ts_tablet_manager.cc:1950] T 7000d25f3bfe4c99be54a0f4789cc96c P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:13.150573 2783 raft_consensus.cc:515] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.151289 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 269) took 5102 ms. Trace:
W20260810 07:53:13.154117 32470 catalog_manager.cc:5979] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1fe71e69dfd74a3b826050bd7edc6b6a consensus state has no leader
I20260810 07:53:13.154431 2783 leader_election.cc:290] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:13.155351 32470 master_service.cc:960] ReplaceTablet: received request to replace tablet 8779af4159eb48da894e6a92c2c4a6f2 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:13.156854 2397 rpcz_store.cc:276] 0810 07:53:08.048579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.049141 (+ 562us) service_pool.cc:224] Handling call
0810 07:53:13.021472 (+4972331us) ts_tablet_manager.cc:1151] Deleting tablet 7000d25f3bfe4c99be54a0f4789cc96c
0810 07:53:13.021531 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.092415 (+ 70884us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.132874 (+ 40459us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.151213 (+ 18339us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4971667}
I20260810 07:53:13.158192 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 7000d25f3bfe4c99be54a0f4789cc96c (table default.Table_d0c496eafbd142de89df899f2a939e6f [id=5dca352fb28a4d16a30d40ad9763aad6]) successfully deleted
I20260810 07:53:13.158727 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 272) took 4297 ms. Trace:
I20260810 07:53:13.159054 2397 rpcz_store.cc:276] 0810 07:53:08.873322 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:08.873460 (+ 138us) service_pool.cc:224] Handling call
0810 07:53:13.158318 (+4284858us) ts_tablet_manager.cc:1151] Deleting tablet 6a97561b5ff5490890e756ca5ff50d38
0810 07:53:13.158370 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.158661 (+ 291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4272931}
I20260810 07:53:13.161695 2397 tablet_replica.cc:333] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:13.162758 2397 raft_consensus.cc:2243] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:13.163436 2397 raft_consensus.cc:2272] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.165695 2397 ts_tablet_manager.cc:1916] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:13.167068 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 6a97561b5ff5490890e756ca5ff50d38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a97561b5ff5490890e756ca5ff50d38 already in progress: creating tablet
I20260810 07:53:13.178635 2624 leader_election.cc:290] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:13.179931 1064 ts_tablet_manager.cc:1434] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.055s user 0.010s sys 0.000s
I20260810 07:53:13.189105 2426 consensus_queue.cc:1048] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:13.192037 32565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
I20260810 07:53:13.052772 2417 ts_tablet_manager.cc:1916] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.206060 1047 tablet_bootstrap.cc:654] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:13.219488 32565 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111)
I20260810 07:53:13.220065 32565 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:13.224617 677 raft_consensus.cc:2468] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:13.170292 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56643.7us
W20260810 07:53:13.231701 573 leader_election.cc:343] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:13.234867 1064 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:13.251220 573 leader_election.cc:343] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:13.252333 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 144219us
I20260810 07:53:13.254390 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_d4eb4954a123470aad7f2107c6680c2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:13.256687 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4eb4954a123470aad7f2107c6680c2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:13.258754 2713 raft_consensus.cc:493] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.259248 2713 raft_consensus.cc:515] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.261384 2713 leader_election.cc:290] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:13.261067 2417 ts_tablet_manager.cc:1929] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260810 07:53:13.262518 573 consensus_peers.cc:597] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:13.264411 2417 log.cc:1199] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/d301af14299040528cf3a033ff5177f8
W20260810 07:53:13.268636 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 77652.2us
I20260810 07:53:13.270767 2417 ts_tablet_manager.cc:1950] T d301af14299040528cf3a033ff5177f8 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:13.272235 308 leader_election.cc:343] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:13.272744 308 leader_election.cc:304] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:13.276238 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 333) took 2347 ms. Trace:
I20260810 07:53:13.278357 2417 rpcz_store.cc:276] 0810 07:53:10.928252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.928821 (+ 569us) service_pool.cc:224] Handling call
0810 07:53:13.029534 (+2100713us) ts_tablet_manager.cc:1151] Deleting tablet d301af14299040528cf3a033ff5177f8
0810 07:53:13.029590 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.204505 (+174915us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.248713 (+ 44208us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.275816 (+ 27103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18196,"tablet-delete.queue_time_us":2096301}
I20260810 07:53:13.293695 2417 tablet_replica.cc:333] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:13.305720 2417 raft_consensus.cc:2243] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:13.313380 2417 raft_consensus.cc:2272] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.320346 2417 ts_tablet_manager.cc:1916] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.329738 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ab03facf8477eada5d1ac3ec7f5ef" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:13.332962 1047 tablet_bootstrap.cc:492] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:13.333561 1047 ts_tablet_manager.cc:1403] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.509s user 0.007s sys 0.004s
I20260810 07:53:13.335493 1292 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.126:38193
I20260810 07:53:13.336766 2825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.126:38193 every 8 connection(s)
I20260810 07:53:13.364395 2417 ts_tablet_manager.cc:1929] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:13.367447 2417 log.cc:1199] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/0098aaa7be644c9885167dc1ee4929b5
I20260810 07:53:13.299281 2397 ts_tablet_manager.cc:1929] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:13.370113 2397 log.cc:1199] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:13.371049 2397 ts_tablet_manager.cc:1950] T c292cb975c504ad8abf03e913140c7a9 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:13.345438 1071 tablet_bootstrap.cc:654] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:13.372903 2417 ts_tablet_manager.cc:1950] T 0098aaa7be644c9885167dc1ee4929b5 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:13.375442 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 337) took 1902 ms. Trace:
I20260810 07:53:13.374590 1292 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
I20260810 07:53:13.375821 2417 rpcz_store.cc:276] 0810 07:53:11.473295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.473422 (+ 127us) service_pool.cc:224] Handling call
0810 07:53:13.292226 (+1818804us) ts_tablet_manager.cc:1151] Deleting tablet 0098aaa7be644c9885167dc1ee4929b5
0810 07:53:13.292701 (+ 475us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.347111 (+ 54410us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.364205 (+ 17094us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.375369 (+ 11164us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1802387}
I20260810 07:53:13.377383 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 338) took 1903 ms. Trace:
I20260810 07:53:13.377728 2417 rpcz_store.cc:276] 0810 07:53:11.473566 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.473778 (+ 212us) service_pool.cc:224] Handling call
0810 07:53:13.377012 (+1903234us) ts_tablet_manager.cc:1151] Deleting tablet c292cb975c504ad8abf03e913140c7a9
0810 07:53:13.377078 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.377308 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1887706}
I20260810 07:53:13.378763 2417 tablet_replica.cc:333] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:13.379601 2417 raft_consensus.cc:2243] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:13.380097 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 1292
I20260810 07:53:13.391808 2713 raft_consensus.cc:2749] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:13.401276 1071 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:13.402015 1071 ts_tablet_manager.cc:1403] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.450s user 0.016s sys 0.002s
I20260810 07:53:13.408193 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet d301af14299040528cf3a033ff5177f8 (table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]) successfully deleted
I20260810 07:53:13.380709 2417 raft_consensus.cc:2272] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.409601 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 0098aaa7be644c9885167dc1ee4929b5 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]) successfully deleted
I20260810 07:53:13.411881 2417 ts_tablet_manager.cc:1916] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:13.410892 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet c292cb975c504ad8abf03e913140c7a9 because the tablet was not found. No further retry: Not found: Tablet not found: c292cb975c504ad8abf03e913140c7a9
I20260810 07:53:13.413897 1071 raft_consensus.cc:359] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:13.417924 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 145711us
I20260810 07:53:13.418345 1071 raft_consensus.cc:385] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:13.420166 2713 raft_consensus.cc:493] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:13.420748 2713 raft_consensus.cc:515] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.422930 2713 leader_election.cc:290] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:13.419816 1071 raft_consensus.cc:740] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:13.424528 1071 consensus_queue.cc:260] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.433228 371 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57540 (request call id 306) took 1114 ms. Trace:
I20260810 07:53:13.433761 371 rpcz_store.cc:276] 0810 07:53:12.319064 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.320503 (+ 1439us) service_pool.cc:224] Handling call
0810 07:53:12.333911 (+ 13408us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:12.334026 (+ 115us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:13.013266 (+679240us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.281990 (+268724us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:13.433168 (+151178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:13.434998 369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57540 (request call id 307) took 1085 ms. Trace:
I20260810 07:53:13.435379 369 rpcz_store.cc:276] 0810 07:53:12.350069 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.350348 (+ 279us) service_pool.cc:224] Handling call
0810 07:53:12.351595 (+ 1247us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:12.351677 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:12.895290 (+543613us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.326008 (+430718us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:13.434946 (+108938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:13.436580 385 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57540 (request call id 304) took 1188 ms. Trace:
I20260810 07:53:13.436702 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 273) took 4250 ms. Trace:
I20260810 07:53:13.436985 385 rpcz_store.cc:276] 0810 07:53:12.247917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.248030 (+ 113us) service_pool.cc:224] Handling call
0810 07:53:12.281310 (+ 33280us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0810 07:53:12.281409 (+ 99us) ts_tablet_manager.cc:682] Creating new metadata...
0810 07:53:12.890126 (+608717us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.354330 (+464204us) tablet_replica.cc:206] Creating consensus instance
0810 07:53:13.436521 (+ 82191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:13.437055 2397 rpcz_store.cc:276] 0810 07:53:09.186625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.186742 (+ 117us) service_pool.cc:224] Handling call
0810 07:53:13.161487 (+3974745us) ts_tablet_manager.cc:1151] Deleting tablet c292cb975c504ad8abf03e913140c7a9
0810 07:53:13.161545 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.237626 (+ 76081us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.298465 (+ 60839us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.436647 (+138182us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3974122}
I20260810 07:53:13.438479 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 274) took 4250 ms. Trace:
I20260810 07:53:13.438760 2397 rpcz_store.cc:276] 0810 07:53:09.187699 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.187797 (+ 98us) service_pool.cc:224] Handling call
0810 07:53:13.438115 (+4250318us) ts_tablet_manager.cc:1151] Deleting tablet 77fa93c49cb54e5fa03034eda6fc2300
0810 07:53:13.438153 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.438424 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4249762}
I20260810 07:53:13.440080 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 275) took 3561 ms. Trace:
I20260810 07:53:13.440442 2397 rpcz_store.cc:276] 0810 07:53:09.878592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:09.879241 (+ 649us) service_pool.cc:224] Handling call
0810 07:53:13.439730 (+3560489us) ts_tablet_manager.cc:1151] Deleting tablet e6800738970d47dd8e7cba68945042aa
0810 07:53:13.439767 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.440021 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3559908}
I20260810 07:53:13.442058 2397 tablet_replica.cc:333] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:13.442863 2397 raft_consensus.cc:2243] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:13.443420 2397 raft_consensus.cc:2272] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.444999 2777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:53:13.448582 2397 ts_tablet_manager.cc:1916] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.464385 2417 ts_tablet_manager.cc:1929] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:13.464897 2417 log.cc:1199] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/213fa4838fe847de87091cfba97eaedc
I20260810 07:53:13.465808 2417 ts_tablet_manager.cc:1950] T 213fa4838fe847de87091cfba97eaedc P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:13.471478 1071 ts_tablet_manager.cc:1434] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.069s user 0.007s sys 0.000s
I20260810 07:53:13.478668 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 339) took 2004 ms. Trace:
I20260810 07:53:13.479095 2417 rpcz_store.cc:276] 0810 07:53:11.473835 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.473996 (+ 161us) service_pool.cc:224] Handling call
0810 07:53:13.378592 (+1904596us) ts_tablet_manager.cc:1151] Deleting tablet 213fa4838fe847de87091cfba97eaedc
0810 07:53:13.378647 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.449254 (+ 70607us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.464211 (+ 14957us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.478588 (+ 14377us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1888854}
I20260810 07:53:13.481467 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet c292cb975c504ad8abf03e913140c7a9 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279]) successfully deleted
W20260810 07:53:13.482537 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 77fa93c49cb54e5fa03034eda6fc2300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77fa93c49cb54e5fa03034eda6fc2300 already in progress: creating tablet
W20260810 07:53:13.483503 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet e6800738970d47dd8e7cba68945042aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6800738970d47dd8e7cba68945042aa already in progress: creating tablet
I20260810 07:53:13.508498 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 329) took 1090 ms. Trace:
I20260810 07:53:13.509065 32464 rpcz_store.cc:276] 0810 07:53:12.418136 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.166433 (+748297us) service_pool.cc:224] Handling call
0810 07:53:13.254274 (+ 87841us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186c1c6 00007f30cf420034 00007f30cf41ffc2 00007f30cf41fef1 00007f30cf41fe6c 00007f30cf41f122 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:13.257693 (+ 3419us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:13.259407 (+ 1714us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:13.506960 (+247553us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:13.507656 (+ 696us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:13.508260 (+ 604us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:13.260798 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:13.261379 (+ 581us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:13.261418 (+ 39us) write_op.cc:435] Acquired schema lock
0810 07:53:13.261440 (+ 22us) tablet.cc:662] Decoding operations
0810 07:53:13.261859 (+ 419us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:13.261917 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:13.261954 (+ 37us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:13.262240 (+ 286us) tablet.cc:701] Row locks acquired
0810 07:53:13.262263 (+ 23us) write_op.cc:260] PREPARE: finished
0810 07:53:13.262415 (+ 152us) write_op.cc:270] Start()
0810 07:53:13.262536 (+ 121us) write_op.cc:276] Timestamp: P: 1786348393262349 usec, L: 0
0810 07:53:13.262561 (+ 25us) op_driver.cc:348] REPLICATION: starting
0810 07:53:13.263169 (+ 608us) log.cc:844] Serialized 433 byte log entry
0810 07:53:13.503949 (+240780us) op_driver.cc:464] REPLICATION: finished
0810 07:53:13.504313 (+ 364us) write_op.cc:301] APPLY: starting
0810 07:53:13.504452 (+ 139us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:13.504617 (+ 165us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:13.504786 (+ 169us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:13.505359 (+ 573us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:13.505514 (+ 155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:13.505576 (+ 62us) write_op.cc:312] APPLY: finished
0810 07:53:13.505777 (+ 201us) log.cc:844] Serialized 29 byte log entry
0810 07:53:13.505941 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.506044 (+ 103us) write_op.cc:454] Released schema lock
0810 07:53:13.506080 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:13.506441 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.506472 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":199595776,"child_traces":[["op",{"apply.queue_time_us":141,"apply.run_cpu_time_us":2058,"apply.run_wall_time_us":2058,"num_ops":2,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":2626,"prepare.run_wall_time_us":2627,"replication_time_us":241224}]]}
I20260810 07:53:13.516927 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_0ed10cb37e024390a3b1b7960835aff7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:13.523537 2417 tablet_replica.cc:333] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
W20260810 07:53:13.524304 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ed10cb37e024390a3b1b7960835aff7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:13.526710 2783 raft_consensus.cc:493] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.527345 2783 raft_consensus.cc:515] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.524585 2417 raft_consensus.cc:2243] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 LEADER]: Raft consensus shutting down.
I20260810 07:53:13.533738 2417 raft_consensus.cc:2272] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.538055 500 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:13.541592 2417 ts_tablet_manager.cc:1916] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.562878 1064 tablet_bootstrap.cc:700] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent opening tablet: real 0.323s user 0.004s sys 0.000s
I20260810 07:53:13.563654 1064 tablet_bootstrap.cc:654] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:13.567842 309 consensus_peers.cc:597] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d301af14299040528cf3a033ff5177f8. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:13.576344 2417 ts_tablet_manager.cc:1929] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:53:13.581753 2417 log.cc:1199] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/ef32f7951f8640e9a13b069816b1cdd5
I20260810 07:53:13.582836 2417 ts_tablet_manager.cc:1950] T ef32f7951f8640e9a13b069816b1cdd5 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:13.585340 2783 leader_election.cc:290] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:13.588577 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 335574us
W20260810 07:53:13.589869 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1fe71e69dfd74a3b826050bd7edc6b6a consensus state has no leader
I20260810 07:53:13.585316 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 336) took 2112 ms. Trace:
W20260810 07:53:13.591532 32452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 61845.5us
I20260810 07:53:13.590963 2417 rpcz_store.cc:276] 0810 07:53:11.473015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.535175 (+ 62160us) service_pool.cc:224] Handling call
0810 07:53:13.480238 (+1945063us) ts_tablet_manager.cc:1151] Deleting tablet ef32f7951f8640e9a13b069816b1cdd5
0810 07:53:13.480303 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.550283 (+ 69980us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.576184 (+ 25901us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.585242 (+ 9058us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1915076}
I20260810 07:53:13.593444 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 340) took 2118 ms. Trace:
I20260810 07:53:13.593847 2417 rpcz_store.cc:276] 0810 07:53:11.608757 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.609860 (+ 1103us) service_pool.cc:224] Handling call
0810 07:53:13.593095 (+1983235us) ts_tablet_manager.cc:1151] Deleting tablet 7eeabaae541548afab4a3ff5ab388035
0810 07:53:13.593152 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.593385 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1947983}
I20260810 07:53:13.595043 2417 tablet_replica.cc:333] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:13.597754 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 213fa4838fe847de87091cfba97eaedc (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]) successfully deleted
I20260810 07:53:13.599102 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]) successfully deleted
W20260810 07:53:13.600431 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because the tablet was not found. No further retry: Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:13.620057 2834 sys_catalog.cc:263] Verifying existing consensus state
W20260810 07:53:13.628103 308 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:13.629390 308 consensus_peers.cc:597] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d301af14299040528cf3a033ff5177f8. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:13.642863 1047 raft_consensus.cc:359] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.643729 1047 raft_consensus.cc:385] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:13.644052 1047 raft_consensus.cc:740] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:13.644936 1047 consensus_queue.cc:260] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.654412 2834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Bootstrap starting.
I20260810 07:53:13.658483 32470 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 8779af4159eb48da894e6a92c2c4a6f2
I20260810 07:53:13.660467 385 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:13.662551 32470 catalog_manager.cc:6807] ReplaceTablet: tablet 8779af4159eb48da894e6a92c2c4a6f2 deleted and replaced by tablet 5a0f23d5212a46cc8f731d9c08019858
I20260810 07:53:13.662636 855 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:13.663012 32470 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 328) took 1285 ms. Trace:
I20260810 07:53:13.673942 2845 tablet_replica.cc:333] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:13.472668 1071 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:13.680071 2834 log.cc:826] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Log is configured to *not* fsync() on all Append() calls
I20260810 07:53:13.687834 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:13.717159 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a3fe9686e0445187c77cf41c563c01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:13.718592 1064 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:13.719120 1064 ts_tablet_manager.cc:1403] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.486s user 0.011s sys 0.000s
W20260810 07:53:13.725836 309 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:13.726308 309 leader_election.cc:304] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:13.728475 1047 ts_tablet_manager.cc:1434] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.394s user 0.005s sys 0.000s
I20260810 07:53:13.756001 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 330) took 1238 ms. Trace:
I20260810 07:53:13.773165 1064 raft_consensus.cc:359] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.773844 1064 raft_consensus.cc:385] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:53:13.774029 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 48235.2us
I20260810 07:53:13.774107 1064 raft_consensus.cc:740] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:13.774667 1064 consensus_queue.cc:260] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:13.775687 309 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: a04e29d3a49b4bd891be79e0fad6ec51
I20260810 07:53:13.780920 2426 raft_consensus.cc:2749] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:13.794438 2397 ts_tablet_manager.cc:1929] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:13.794837 2397 log.cc:1199] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/a04e29d3a49b4bd891be79e0fad6ec51
I20260810 07:53:13.795579 2397 ts_tablet_manager.cc:1950] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:13.800832 1047 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:13.810932 32464 rpcz_store.cc:276] 0810 07:53:12.517286 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.516326 (+999040us) service_pool.cc:224] Handling call
0810 07:53:13.524790 (+ 8464us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:13.648338 (+123548us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:13.755344 (+107006us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:13.755657 (+ 313us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:13.755923 (+ 266us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:13.729071 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:13.729497 (+ 426us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:13.729520 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:13.729536 (+ 16us) tablet.cc:662] Decoding operations
0810 07:53:13.729842 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:13.729885 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:13.729908 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:13.730105 (+ 197us) tablet.cc:701] Row locks acquired
0810 07:53:13.730119 (+ 14us) write_op.cc:260] PREPARE: finished
0810 07:53:13.730224 (+ 105us) write_op.cc:270] Start()
0810 07:53:13.730315 (+ 91us) write_op.cc:276] Timestamp: P: 1786348393730131 usec, L: 0
0810 07:53:13.730330 (+ 15us) op_driver.cc:348] REPLICATION: starting
0810 07:53:13.730858 (+ 528us) log.cc:844] Serialized 433 byte log entry
0810 07:53:13.740698 (+ 9840us) op_driver.cc:464] REPLICATION: finished
0810 07:53:13.744502 (+ 3804us) write_op.cc:301] APPLY: starting
0810 07:53:13.744555 (+ 53us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:13.744658 (+ 103us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:13.745169 (+ 511us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:13.745851 (+ 682us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:13.746845 (+ 994us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:13.746889 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:13.747077 (+ 188us) log.cc:844] Serialized 29 byte log entry
0810 07:53:13.747254 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.747329 (+ 75us) write_op.cc:454] Released schema lock
0810 07:53:13.747359 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:13.747617 (+ 258us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.747642 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3604,"apply.run_cpu_time_us":1659,"apply.run_wall_time_us":3098,"num_ops":2,"prepare.queue_time_us":79111,"prepare.run_cpu_time_us":2091,"prepare.run_wall_time_us":2093,"raft.queue_time_us":187,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":663,"replication_time_us":10246}]]}
W20260810 07:53:13.834842 308 consensus_peers.cc:597] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9be900b701a45728a44bf26350a26ab. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:13.836942 2799 raft_consensus.cc:493] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.837769 1064 ts_tablet_manager.cc:1434] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.118s user 0.005s sys 0.000s
I20260810 07:53:13.838706 1064 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:13.837810 2799 raft_consensus.cc:515] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.842592 2799 leader_election.cc:290] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:13.846596 1064 tablet_bootstrap.cc:654] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:13.851853 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 282) took 3171 ms. Trace:
I20260810 07:53:13.852202 2397 rpcz_store.cc:276] 0810 07:53:10.680203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.680712 (+ 509us) service_pool.cc:224] Handling call
0810 07:53:13.441924 (+2761212us) ts_tablet_manager.cc:1151] Deleting tablet a04e29d3a49b4bd891be79e0fad6ec51
0810 07:53:13.441961 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:13.698593 (+256632us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.794309 (+ 95716us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:13.851780 (+ 57471us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2760577}
I20260810 07:53:13.854681 2397 tablet_replica.cc:333] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:13.855492 2397 raft_consensus.cc:2243] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:13.862970 351 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:57532 (request call id 44) took 1110 ms. Trace:
I20260810 07:53:13.863327 351 rpcz_store.cc:276] 0810 07:53:12.752401 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.752526 (+ 125us) service_pool.cc:224] Handling call
0810 07:53:13.862901 (+1110375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:13.869087 2619 raft_consensus.cc:493] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.869530 2619 raft_consensus.cc:515] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.871713 2619 leader_election.cc:290] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:13.884940 308 consensus_peers.cc:597] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:13.901193 308 leader_election.cc:343] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: a04e29d3a49b4bd891be79e0fad6ec51
W20260810 07:53:13.901993 800 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 57814.9us
I20260810 07:53:13.910514 308 leader_election.cc:304] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:13.911417 308 raft_consensus.cc:2686] T a04e29d3a49b4bd891be79e0fad6ec51 P 60781ef3c8b84a8788f43d77e1bb3b19: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260810 07:53:13.913254 308 consensus_peers.cc:597] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0098aaa7be644c9885167dc1ee4929b5. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:13.853657 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet a04e29d3a49b4bd891be79e0fad6ec51 (table default.Table_71d864751edf4f7786f4aea1d9e5e8ad [id=ad1d545c245b45d19afe4cd8816fa3fe]) successfully deleted
W20260810 07:53:13.957059 32450 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 103448us
I20260810 07:53:13.959317 2799 raft_consensus.cc:493] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.959831 2799 raft_consensus.cc:515] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:13.961922 2799 leader_election.cc:290] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:13.663327 32470 rpcz_store.cc:276] 0810 07:53:12.377554 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.155213 (+777659us) service_pool.cc:224] Handling call
0810 07:53:13.253260 (+ 98047us) spinlock_profiling.cc:243] Waited 96.4 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ed41 00007f30cf440d37 00007f30cf440cc2 00007f30cf440bf1 00007f30cf440b6c 00007f30cf43fe22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:13.662904 (+409644us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:13.497305 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:13.497798 (+ 493us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:13.497833 (+ 35us) write_op.cc:435] Acquired schema lock
0810 07:53:13.497854 (+ 21us) tablet.cc:662] Decoding operations
0810 07:53:13.498259 (+ 405us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:13.498326 (+ 67us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:13.498353 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:13.498642 (+ 289us) tablet.cc:701] Row locks acquired
0810 07:53:13.498666 (+ 24us) write_op.cc:260] PREPARE: finished
0810 07:53:13.498790 (+ 124us) write_op.cc:270] Start()
0810 07:53:13.498892 (+ 102us) write_op.cc:276] Timestamp: P: 1786348393498704 usec, L: 0
0810 07:53:13.498909 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:13.499608 (+ 699us) log.cc:844] Serialized 662 byte log entry
0810 07:53:13.536590 (+ 36982us) op_driver.cc:464] REPLICATION: finished
0810 07:53:13.537856 (+ 1266us) write_op.cc:301] APPLY: starting
0810 07:53:13.544845 (+ 6989us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:13.545185 (+ 340us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:13.545294 (+ 109us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:13.545786 (+ 492us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:13.546021 (+ 235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:13.546083 (+ 62us) write_op.cc:312] APPLY: finished
0810 07:53:13.546303 (+ 220us) log.cc:844] Serialized 29 byte log entry
0810 07:53:13.546521 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.546606 (+ 85us) write_op.cc:454] Released schema lock
0810 07:53:13.546649 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:13.547265 (+ 616us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.547319 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":221791616,"child_traces":[["op",{"apply.queue_time_us":1088,"apply.run_cpu_time_us":2005,"apply.run_wall_time_us":9294,"num_ops":2,"prepare.queue_time_us":163,"prepare.run_cpu_time_us":2533,"prepare.run_wall_time_us":2533,"replication_time_us":37572}]]}
I20260810 07:53:13.979805 1047 tablet_bootstrap.cc:654] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:13.981081 32464 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_09d2a4538e6c41de842dbd63da2067f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:13.982962 32464 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09d2a4538e6c41de842dbd63da2067f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:13.987888 2799 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:13.988488 2799 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:13.990634 2799 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:14.007862 2868 tablet.cc:1724] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:14.023188 1064 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:14.023778 1064 ts_tablet_manager.cc:1403] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.185s user 0.023s sys 0.040s
I20260810 07:53:14.029011 2845 raft_consensus.cc:2243] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:14.029604 2845 raft_consensus.cc:2272] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.916249 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]). Expected version 1 got 0
I20260810 07:53:14.031275 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:14.031646 2845 ts_tablet_manager.cc:1916] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:14.032039 32468 catalog_manager.cc:5719] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:14.033421 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:14.034960 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:14.035882 32468 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]). Expected version 1 got 0
I20260810 07:53:13.900581 1071 tablet_bootstrap.cc:654] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:13.890365 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ab03facf8477eada5d1ac3ec7f5ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.045022 413 raft_consensus.cc:2468] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:13.868634 2397 raft_consensus.cc:2272] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:13.924556 2853 consensus_queue.cc:1048] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:14.055018 2397 ts_tablet_manager.cc:1916] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:13.950587 2865 tablet.cc:1724] T 82edc39a864f49f8be60e4b36ce3d59c P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:13.992427 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:13.994714 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:14.064208 385 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57540: Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:13.998409 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:13.999879 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:14.067085 32450 catalog_manager.cc:5129] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): alter failed for tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]): Illegal state: replica 60781ef3c8b84a8788f43d77e1bb3b19 is not leader of this config: current role FOLLOWER
I20260810 07:53:14.080008 1064 raft_consensus.cc:359] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:14.080770 1064 raft_consensus.cc:385] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.081063 1064 raft_consensus.cc:740] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.081727 1064 consensus_queue.cc:260] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:14.092396 1064 ts_tablet_manager.cc:1434] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.068s user 0.005s sys 0.000s
I20260810 07:53:14.100804 2845 ts_tablet_manager.cc:1929] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:14.114753 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.116935 2845 log.cc:1199] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/8779af4159eb48da894e6a92c2c4a6f2
I20260810 07:53:14.118022 2845 ts_tablet_manager.cc:1950] T 8779af4159eb48da894e6a92c2c4a6f2 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:14.121132 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]) successfully deleted
I20260810 07:53:14.129216 1064 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:13.995496 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:13.997108 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.001073 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.147913 1071 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.003496 2417 raft_consensus.cc:2243] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:14.149119 2417 raft_consensus.cc:2272] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:14.149726 1071 ts_tablet_manager.cc:1403] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.677s user 0.009s sys 0.002s
I20260810 07:53:14.151602 2417 ts_tablet_manager.cc:1916] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:14.046556 573 leader_election.cc:304] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:14.152406 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 105959us
W20260810 07:53:14.153443 573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:14.065193 742 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:14.104990 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.125571 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.154206 1071 raft_consensus.cc:359] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.161464 1071 raft_consensus.cc:385] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.161767 1071 raft_consensus.cc:740] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.164919 2619 raft_consensus.cc:2804] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:14.167227 1071 consensus_queue.cc:260] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.167625 2619 raft_consensus.cc:493] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:14.167996 2619 raft_consensus.cc:3060] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.169754 1071 ts_tablet_manager.cc:1434] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260810 07:53:14.171203 2713 raft_consensus.cc:493] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:14.171792 2713 raft_consensus.cc:515] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.174960 2713 leader_election.cc:290] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:14.177536 1071 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:14.168592 573 consensus_peers.cc:597] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:14.187830 963 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:14.191547 1071 tablet_bootstrap.cc:654] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.176461 2619 raft_consensus.cc:515] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.208114 574 leader_election.cc:304] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:14.219792 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:14.220439 892 raft_consensus.cc:2468] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:14.221653 309 leader_election.cc:304] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:14.222492 2713 raft_consensus.cc:2804] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:14.222849 2713 raft_consensus.cc:493] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:14.223191 2713 raft_consensus.cc:3060] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.235265 2713 raft_consensus.cc:515] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:14.249773 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 336599us
I20260810 07:53:14.270438 1047 tablet_bootstrap.cc:492] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:14.270961 1047 ts_tablet_manager.cc:1403] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.470s user 0.010s sys 0.000s
I20260810 07:53:14.233966 2619 leader_election.cc:290] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:14.273584 1047 raft_consensus.cc:359] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.274221 1047 raft_consensus.cc:385] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.274502 1047 raft_consensus.cc:740] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.274842 2619 raft_consensus.cc:2749] T 45a3fe9686e0445187c77cf41c563c01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:14.275179 1047 consensus_queue.cc:260] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.276757 1047 ts_tablet_manager.cc:1434] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:53:14.278662 1047 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:14.278975 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 70947.1us
I20260810 07:53:14.282823 1064 tablet_bootstrap.cc:654] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:14.282760 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 89953us
I20260810 07:53:14.287686 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ab03facf8477eada5d1ac3ec7f5ef" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:14.288252 414 raft_consensus.cc:3060] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.292876 1047 tablet_bootstrap.cc:654] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.298350 414 raft_consensus.cc:2468] T 2d5ab03facf8477eada5d1ac3ec7f5ef P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:14.307159 573 leader_election.cc:304] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:14.310810 2872 raft_consensus.cc:2804] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:14.311265 2872 raft_consensus.cc:697] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:14.313347 1047 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:14.313773 1047 ts_tablet_manager.cc:1403] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.005s
I20260810 07:53:14.316210 1047 raft_consensus.cc:359] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.316848 1047 raft_consensus.cc:385] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.317129 1047 raft_consensus.cc:740] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.316979 2872 consensus_queue.cc:237] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.317708 1047 consensus_queue.cc:260] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.319164 1047 ts_tablet_manager.cc:1434] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:53:14.320019 1047 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:14.322088 2713 leader_election.cc:290] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:14.323711 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:14.324301 892 raft_consensus.cc:3060] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.332815 892 raft_consensus.cc:2468] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:14.333607 1047 tablet_bootstrap.cc:654] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.335115 309 leader_election.cc:304] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:14.337170 2853 raft_consensus.cc:2804] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:14.337705 2853 raft_consensus.cc:697] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:14.338568 2853 consensus_queue.cc:237] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.354507 2417 ts_tablet_manager.cc:1929] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:14.355057 2417 log.cc:1199] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/3fe822aa30b54fb091fd62ed8b6e22ff
I20260810 07:53:14.355985 2417 ts_tablet_manager.cc:1950] T 3fe822aa30b54fb091fd62ed8b6e22ff P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:14.361195 2877 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:14.361686 2877 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:14.364218 2877 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:14.366410 1064 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:14.366839 1064 ts_tablet_manager.cc:1403] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.238s user 0.008s sys 0.003s
I20260810 07:53:14.369333 1064 raft_consensus.cc:359] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:14.369959 1064 raft_consensus.cc:385] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.370301 1064 raft_consensus.cc:740] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.371120 1064 consensus_queue.cc:260] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:14.372412 574 consensus_peers.cc:597] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:14.373958 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.374650 1071 tablet_bootstrap.cc:492] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.376935 1071 ts_tablet_manager.cc:1403] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.200s user 0.012s sys 0.033s
I20260810 07:53:14.385067 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.390580 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 343) took 2742 ms. Trace:
I20260810 07:53:14.401648 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.402736 667 raft_consensus.cc:2468] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:14.406293 1064 ts_tablet_manager.cc:1434] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260810 07:53:14.407711 1064 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:14.391215 2417 rpcz_store.cc:276] 0810 07:53:11.647847 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.648025 (+ 178us) service_pool.cc:224] Handling call
0810 07:53:13.594863 (+1946838us) ts_tablet_manager.cc:1151] Deleting tablet 3fe822aa30b54fb091fd62ed8b6e22ff
0810 07:53:13.594918 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:14.205340 (+610422us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.354320 (+148980us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.390485 (+ 36165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":404574,"tablet-delete.queue_time_us":1946317}
I20260810 07:53:14.426518 1071 raft_consensus.cc:359] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.427379 1071 raft_consensus.cc:385] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.427671 1071 raft_consensus.cc:740] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.440943 1071 consensus_queue.cc:260] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.465273 1064 tablet_bootstrap.cc:654] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.469524 2834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=1 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260810 07:53:14.472110 2834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Bootstrap complete.
I20260810 07:53:14.475651 1071 ts_tablet_manager.cc:1434] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.097s user 0.007s sys 0.000s
I20260810 07:53:14.476468 2632 consensus_queue.cc:1048] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:14.476704 1071 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:14.486749 1071 tablet_bootstrap.cc:654] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:14.496246 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 77080.3us
W20260810 07:53:14.497125 574 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:14.498402 2417 tablet_replica.cc:333] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:14.515122 1064 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:14.515846 1064 ts_tablet_manager.cc:1403] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.108s user 0.013s sys 0.002s
I20260810 07:53:14.519429 2899 tablet.cc:1724] T 82edc39a864f49f8be60e4b36ce3d59c P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:14.525342 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.526434 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.529703 1064 raft_consensus.cc:359] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:14.533303 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 398 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:14.539065 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371fa2534984499b95c0945af358e8fc" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:14.539669 664 raft_consensus.cc:3060] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.567242 664 raft_consensus.cc:2468] T 371fa2534984499b95c0945af358e8fc P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:14.576516 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.586326 1071 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.586853 1071 ts_tablet_manager.cc:1403] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.110s user 0.013s sys 0.003s
I20260810 07:53:14.589952 1071 raft_consensus.cc:359] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.590794 1071 raft_consensus.cc:385] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.591071 1071 raft_consensus.cc:740] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.592875 1071 consensus_queue.cc:260] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.594707 1071 ts_tablet_manager.cc:1434] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260810 07:53:14.600400 1071 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:14.603873 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 3fe822aa30b54fb091fd62ed8b6e22ff (table default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]) successfully deleted
I20260810 07:53:14.607074 1064 raft_consensus.cc:385] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.620544 1071 tablet_bootstrap.cc:654] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.625059 1064 raft_consensus.cc:740] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:14.624395 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 148646us
I20260810 07:53:14.644572 1071 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.645073 1071 ts_tablet_manager.cc:1403] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.007s
I20260810 07:53:14.657163 1071 raft_consensus.cc:359] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.657866 1071 raft_consensus.cc:385] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.658134 1071 raft_consensus.cc:740] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.659498 1071 consensus_queue.cc:260] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.681113 2865 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57540 (request call id 309) took 1042 ms. Trace:
I20260810 07:53:14.686582 1071 ts_tablet_manager.cc:1434] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260810 07:53:14.687558 1071 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:14.693157 1071 tablet_bootstrap.cc:654] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.637295 1064 consensus_queue.cc:260] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:14.694908 1064 ts_tablet_manager.cc:1434] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.179s user 0.003s sys 0.004s
I20260810 07:53:14.732311 2632 raft_consensus.cc:493] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:14.733422 2632 raft_consensus.cc:515] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.734262 1071 tablet_bootstrap.cc:492] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.734800 1071 ts_tablet_manager.cc:1403] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.047s user 0.017s sys 0.004s
I20260810 07:53:14.737790 2632 leader_election.cc:290] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:14.737654 1071 raft_consensus.cc:359] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.738312 1071 raft_consensus.cc:385] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.738571 1071 raft_consensus.cc:740] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.739701 1071 consensus_queue.cc:260] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.609922 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.617777 2417 raft_consensus.cc:2243] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:14.744943 1071 ts_tablet_manager.cc:1434] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260810 07:53:14.745369 2417 raft_consensus.cc:2272] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:14.747767 2417 ts_tablet_manager.cc:1916] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:14.710748 1047 tablet_bootstrap.cc:492] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
W20260810 07:53:14.716907 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 67949.2us
I20260810 07:53:14.774722 1047 ts_tablet_manager.cc:1403] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.455s user 0.099s sys 0.111s
I20260810 07:53:14.759684 2834 raft_consensus.cc:359] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
W20260810 07:53:14.775144 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55aaf296555b43f9a27f2926054c8d4b consensus state has no leader
I20260810 07:53:14.681687 2865 rpcz_store.cc:276] 0810 07:53:13.638601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.638742 (+ 141us) service_pool.cc:224] Handling call
0810 07:53:14.681028 (+1042286us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:13.652805 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:13.653216 (+ 411us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:13.653245 (+ 29us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:13.653291 (+ 46us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:13.653411 (+ 120us) alter_schema_op.cc:128] START. Timestamp: 1786348393653325
0810 07:53:13.653435 (+ 24us) op_driver.cc:348] REPLICATION: starting
0810 07:53:13.654002 (+ 567us) log.cc:844] Serialized 173 byte log entry
0810 07:53:13.927525 (+273523us) op_driver.cc:464] REPLICATION: finished
0810 07:53:13.950471 (+ 22946us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:14.096099 (+145628us) log.cc:844] Serialized 5 byte log entry
0810 07:53:14.463623 (+367524us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.464618 (+ 995us) log.cc:844] Serialized 14 byte log entry
0810 07:53:14.680838 (+216220us) alter_schema_op.cc:73] Released schema lock
0810 07:53:14.680879 (+ 41us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22731,"mutex_wait_us":411,"prepare.queue_time_us":13182,"prepare.run_cpu_time_us":2012,"prepare.run_wall_time_us":19274,"raft.queue_time_us":245,"raft.run_cpu_time_us":1620,"raft.run_wall_time_us":1621,"replication_time_us":273976,"spinlock_wait_cycles":153344,"thread_start_us":54472,"threads_started":3,"wal-append.queue_time_us":7636}]]}
I20260810 07:53:14.807931 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:14.829336 2397 ts_tablet_manager.cc:1929] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:14.829917 2397 log.cc:1199] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/a9be900b701a45728a44bf26350a26ab
I20260810 07:53:14.830868 2397 ts_tablet_manager.cc:1950] T a9be900b701a45728a44bf26350a26ab P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:14.838306 1071 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:14.842624 308 consensus_peers.cc:597] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:14.842806 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 283) took 4153 ms. Trace:
I20260810 07:53:14.847779 2880 raft_consensus.cc:493] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:14.848322 2880 raft_consensus.cc:515] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.850630 2880 leader_election.cc:290] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:14.844106 2397 rpcz_store.cc:276] 0810 07:53:10.689419 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.691124 (+ 1705us) service_pool.cc:224] Handling call
0810 07:53:13.854234 (+3163110us) ts_tablet_manager.cc:1151] Deleting tablet a9be900b701a45728a44bf26350a26ab
0810 07:53:13.854275 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:14.262454 (+408179us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.829145 (+566691us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.842527 (+ 13382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12262,"tablet-delete.queue_time_us":3162568}
I20260810 07:53:14.857369 1047 raft_consensus.cc:359] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.871208 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 287) took 4003 ms. Trace:
I20260810 07:53:14.874176 2397 rpcz_store.cc:276] 0810 07:53:10.867450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.870473 (+ 3023us) service_pool.cc:224] Handling call
0810 07:53:14.856471 (+3985998us) ts_tablet_manager.cc:1151] Deleting tablet dd22d1029ff249a0a9e05862293261ff
0810 07:53:14.870207 (+ 13736us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:14.870943 (+ 736us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3964354}
I20260810 07:53:14.878604 2397 tablet_replica.cc:333] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:14.893124 1047 raft_consensus.cc:385] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.893505 1047 raft_consensus.cc:740] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.894243 1047 consensus_queue.cc:260] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.895922 1047 ts_tablet_manager.cc:1434] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.121s user 0.003s sys 0.000s
I20260810 07:53:14.903967 2397 raft_consensus.cc:2243] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:14.904891 2397 raft_consensus.cc:2272] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:14.906850 2397 ts_tablet_manager.cc:1916] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:14.907661 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 63784.7us
I20260810 07:53:14.918516 1047 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:14.922991 1071 tablet_bootstrap.cc:654] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.932785 1047 tablet_bootstrap.cc:654] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:14.933104 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet a9be900b701a45728a44bf26350a26ab (table default.Table_9a4d68bb16a74d4786e2176453635dc7 [id=9d40f00f43654e31aa84d19151d0cfc0]) successfully deleted
I20260810 07:53:14.939100 414 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:14.941396 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet dd22d1029ff249a0a9e05862293261ff because the tablet was not found. No further retry: Not found: Tablet not found: dd22d1029ff249a0a9e05862293261ff
I20260810 07:53:14.943169 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:14.943842 410 raft_consensus.cc:2468] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:53:14.952064 308 leader_election.cc:343] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:14.954067 1071 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:14.954519 1071 ts_tablet_manager.cc:1403] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.116s user 0.012s sys 0.010s
I20260810 07:53:14.957988 1071 raft_consensus.cc:359] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.958693 1071 raft_consensus.cc:385] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:14.959026 1071 raft_consensus.cc:740] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:14.959769 1071 consensus_queue.cc:260] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:14.963860 1071 ts_tablet_manager.cc:1434] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.009s user 0.002s sys 0.005s
I20260810 07:53:14.998067 2834 raft_consensus.cc:740] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fde54f23f34224ac756f11924c9531, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.004168 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:14.999536 2834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:15.005268 892 raft_consensus.cc:2468] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:15.002779 2924 raft_consensus.cc:493] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.007130 2924 raft_consensus.cc:515] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.011451 2927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:15.016981 309 leader_election.cc:304] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.020576 2459 raft_consensus.cc:2804] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.021066 2459 raft_consensus.cc:493] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.021121 2927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:15.021451 2459 raft_consensus.cc:3060] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.022120 2924 leader_election.cc:290] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.016230 2834 sys_catalog.cc:565] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:53:15.030821 2459 raft_consensus.cc:515] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.034492 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.035283 891 raft_consensus.cc:3060] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.050376 2924 raft_consensus.cc:493] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.051472 2924 raft_consensus.cc:515] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.057848 2924 leader_election.cc:290] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:15.058372 1064 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:15.060756 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 59797.1us
I20260810 07:53:15.077116 2397 ts_tablet_manager.cc:1929] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:15.077646 2397 log.cc:1199] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/3fe822aa30b54fb091fd62ed8b6e22ff
W20260810 07:53:15.086002 32565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 142851us
I20260810 07:53:15.086923 891 raft_consensus.cc:2468] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:15.088449 309 leader_election.cc:304] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:15.089071 1064 tablet_bootstrap.cc:654] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.089530 2426 raft_consensus.cc:2804] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.089419 2740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 398 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:15.090313 2426 raft_consensus.cc:697] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:15.090754 2740 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 3.
I20260810 07:53:15.091192 2426 consensus_queue.cc:237] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.093039 32565 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531; no voters: d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:15.102780 1071 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.115105 2459 leader_election.cc:290] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:14.945084 573 leader_election.cc:304] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:15.121549 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:15.122159 666 raft_consensus.cc:3060] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:14.965252 2417 ts_tablet_manager.cc:1929] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:15.122678 2417 log.cc:1199] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/45c5e309389d402092f59c801923854a
I20260810 07:53:15.123426 2417 ts_tablet_manager.cc:1950] T 45c5e309389d402092f59c801923854a P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:15.122113 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 177117us
I20260810 07:53:15.127002 2880 raft_consensus.cc:2804] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.127358 2880 raft_consensus.cc:493] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.127661 2880 raft_consensus.cc:3060] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.129613 666 raft_consensus.cc:2468] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:15.137497 2880 raft_consensus.cc:515] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.140986 2397 ts_tablet_manager.cc:1950] T 3fe822aa30b54fb091fd62ed8b6e22ff P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:15.141510 1071 tablet_bootstrap.cc:654] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.141886 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 293) took 4272 ms. Trace:
I20260810 07:53:15.142201 2397 rpcz_store.cc:276] 0810 07:53:10.869227 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.891510 (+ 22283us) service_pool.cc:224] Handling call
0810 07:53:14.877908 (+3986398us) ts_tablet_manager.cc:1151] Deleting tablet 3fe822aa30b54fb091fd62ed8b6e22ff
0810 07:53:14.878169 (+ 261us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:14.929686 (+ 51517us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.076981 (+147295us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.141832 (+ 64851us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23760,"tablet-delete.queue_time_us":3985407}
I20260810 07:53:15.143566 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 3fe822aa30b54fb091fd62ed8b6e22ff (table default.Table_826c5b5c4ab84216af2be3ae4ce22024 [id=e798e55ba6a5436b8ddabef4e258bfb1]) successfully deleted
I20260810 07:53:15.144423 2397 tablet_replica.cc:333] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:15.151268 2880 leader_election.cc:290] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:15.151361 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:15.152292 414 raft_consensus.cc:3060] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.153508 1071 tablet_bootstrap.cc:492] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.157009 1071 ts_tablet_manager.cc:1403] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.006s
I20260810 07:53:15.160806 1071 raft_consensus.cc:359] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.163554 2934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260810 07:53:15.164182 2934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:15.164902 2934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 3 FOLLOWER]: Advancing to term 4
I20260810 07:53:15.169116 1071 raft_consensus.cc:385] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.169891 1071 raft_consensus.cc:740] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.171386 1071 consensus_queue.cc:260] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.177086 1071 ts_tablet_manager.cc:1434] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260810 07:53:15.188756 1071 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.192641 1064 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:15.193423 1064 ts_tablet_manager.cc:1403] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.136s user 0.013s sys 0.010s
I20260810 07:53:15.184448 351 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:57532 (request call id 45) took 1007 ms. Trace:
I20260810 07:53:15.193182 2426 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.194284 2426 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.196256 2426 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.196619 1047 tablet_bootstrap.cc:492] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:15.197224 1047 ts_tablet_manager.cc:1403] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.279s user 0.012s sys 0.000s
I20260810 07:53:15.199102 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.199724 666 raft_consensus.cc:2468] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:15.200126 1071 tablet_bootstrap.cc:654] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.202384 308 leader_election.cc:304] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:15.206147 2911 raft_consensus.cc:2804] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:53:15.206598 2911 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.207005 2911 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:15.209920 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.211144 351 rpcz_store.cc:276] 0810 07:53:14.177130 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:14.177286 (+ 156us) service_pool.cc:224] Handling call
0810 07:53:14.349866 (+172580us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b54000a79a8. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b9c3b41 00007f9c6ba4fd1c 00007f9c6c66d4c0 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:15.096434 (+746568us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b54000a74a8. stack: 00007f9c6446ccf3 00007f9c6446cbf0 00007f9c6401a574 0000000000516de4 0000000000516d79 00007f9c6b9c3b41 00007f9c6ba4fd1c 00007f9c6c66d4c0 00007f9c6adbf104 00007f9c6adbf092 00007f9c6adbefc1 00007f9c6adbef3c 00007f9c6adbe1f2 00007f9c692cf07c 00007f9c692ce4b6 00007f9c692cde5f
0810 07:53:15.184359 (+ 87925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71758976}
I20260810 07:53:15.212838 1071 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.213361 1071 ts_tablet_manager.cc:1403] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.011s
I20260810 07:53:15.216325 1071 raft_consensus.cc:359] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.217104 1071 raft_consensus.cc:385] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.217391 1071 raft_consensus.cc:740] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.218073 1071 consensus_queue.cc:260] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.219110 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.219719 1071 ts_tablet_manager.cc:1434] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:53:15.219820 894 raft_consensus.cc:2468] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:15.222047 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a97561b5ff5490890e756ca5ff50d38" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.222785 896 raft_consensus.cc:3060] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:15.225255 792 leader_election.cc:343] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.225798 1071 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:15.226033 792 leader_election.cc:343] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.232002 1071 tablet_bootstrap.cc:654] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:15.232800 792 leader_election.cc:343] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.237677 2911 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.255044 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 342) took 3645 ms. Trace:
I20260810 07:53:15.255419 2417 rpcz_store.cc:276] 0810 07:53:11.609654 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.644565 (+ 34911us) service_pool.cc:224] Handling call
0810 07:53:14.498170 (+2853605us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:14.498285 (+ 115us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:14.793014 (+294729us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:14.918705 (+125691us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.254957 (+336252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118423,"tablet-delete.queue_time_us":2844836}
I20260810 07:53:15.259222 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 341) took 3647 ms. Trace:
I20260810 07:53:15.259572 2417 rpcz_store.cc:276] 0810 07:53:11.609362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.789340 (+179978us) service_pool.cc:224] Handling call
0810 07:53:15.256463 (+3467123us) ts_tablet_manager.cc:1151] Deleting tablet 7000d25f3bfe4c99be54a0f4789cc96c
0810 07:53:15.256511 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.256683 (+ 172us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3466515}
I20260810 07:53:15.260941 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 352) took 2863 ms. Trace:
I20260810 07:53:15.261018 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 45c5e309389d402092f59c801923854a (table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]) successfully deleted
I20260810 07:53:15.261304 2417 rpcz_store.cc:276] 0810 07:53:12.398009 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.403743 (+ 5734us) service_pool.cc:224] Handling call
0810 07:53:15.260534 (+2856791us) ts_tablet_manager.cc:1151] Deleting tablet 2a8f390b0db64e1d96c3d8686274cbbf
0810 07:53:15.260572 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.260889 (+ 317us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2855943}
I20260810 07:53:15.262207 2417 tablet_replica.cc:333] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:15.262917 2417 raft_consensus.cc:2243] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:15.264394 2417 raft_consensus.cc:2272] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:53:15.266610 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7000d25f3bfe4c99be54a0f4789cc96c because the tablet was not found. No further retry: Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c
W20260810 07:53:15.267303 32450 catalog_manager.cc:5030] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 2a8f390b0db64e1d96c3d8686274cbbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a8f390b0db64e1d96c3d8686274cbbf already in progress: creating tablet
I20260810 07:53:15.276415 1047 raft_consensus.cc:359] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.277308 1047 raft_consensus.cc:385] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.277640 1047 raft_consensus.cc:740] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.266984 2417 ts_tablet_manager.cc:1916] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:15.286375 1064 raft_consensus.cc:359] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.287616 1064 raft_consensus.cc:385] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.288036 1064 raft_consensus.cc:740] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.289209 1064 consensus_queue.cc:260] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:15.234246 792 leader_election.cc:343] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:15.295028 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 60819.1us
W20260810 07:53:15.316181 32565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 223009us
W20260810 07:53:15.324069 792 leader_election.cc:343] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.325033 2919 raft_consensus.cc:493] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.325595 2919 raft_consensus.cc:515] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.258486 2853 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.331368 2853 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.333562 2853 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.335216 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.335830 891 raft_consensus.cc:2468] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:15.336592 2884 consensus_queue.cc:1048] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:15.337174 309 leader_election.cc:304] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:15.265456 2426 raft_consensus.cc:493] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:15.338001 2426 raft_consensus.cc:515] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.339857 2426 leader_election.cc:290] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:15.328246 2919 leader_election.cc:290] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:15.340374 2426 raft_consensus.cc:2804] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.340683 2426 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.290287 2632 raft_consensus.cc:493] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.341394 2397 raft_consensus.cc:2243] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:15.341972 2632 raft_consensus.cc:515] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.342540 2397 raft_consensus.cc:2272] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:15.343854 2632 leader_election.cc:290] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.344432 2397 ts_tablet_manager.cc:1916] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:15.344834 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.347434 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.348397 891 raft_consensus.cc:2468] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:15.349601 309 leader_election.cc:304] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:15.350385 2853 raft_consensus.cc:2804] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.350697 2853 raft_consensus.cc:493] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.350999 2853 raft_consensus.cc:3060] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.344442 2426 raft_consensus.cc:3060] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.279419 414 raft_consensus.cc:2468] T 6a97561b5ff5490890e756ca5ff50d38 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:15.358148 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.361447 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10341e269a934612bb0cf6ef29818a39" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.361955 1047 consensus_queue.cc:260] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:15.363332 308 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.364665 308 leader_election.cc:304] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.364951 1047 ts_tablet_manager.cc:1434] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.167s user 0.008s sys 0.000s
I20260810 07:53:15.365545 2632 raft_consensus.cc:2749] T 10341e269a934612bb0cf6ef29818a39 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:15.365880 1047 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:15.366829 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:15.366983 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:15.368671 308 leader_election.cc:343] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.369850 573 leader_election.cc:304] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:15.370848 2884 raft_consensus.cc:2804] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.371464 2884 raft_consensus.cc:697] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:15.373147 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.372479 2884 consensus_queue.cc:237] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:15.376928 792 leader_election.cc:343] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.380846 2853 raft_consensus.cc:515] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.383610 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55aaf296555b43f9a27f2926054c8d4b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:15.384264 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:15.384307 666 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
W20260810 07:53:15.389420 792 leader_election.cc:343] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:15.395329 792 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.398098 32461 catalog_manager.cc:5811] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:15.401204 32461 catalog_manager.cc:5719] T 45a3fe9686e0445187c77cf41c563c01 P 60781ef3c8b84a8788f43d77e1bb3b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:15.382211 2911 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.382879 2632 raft_consensus.cc:493] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.404588 2632 raft_consensus.cc:515] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.406668 2632 leader_election.cc:290] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.410110 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:15.411717 308 leader_election.cc:343] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.388368 2426 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.424762 2853 leader_election.cc:290] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.426286 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:15.427775 308 leader_election.cc:343] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.431608 2397 ts_tablet_manager.cc:1929] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:15.431978 2397 log.cc:1199] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/45c5e309389d402092f59c801923854a
I20260810 07:53:15.433876 2426 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.436522 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:15.402261 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.408401 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.438066 894 raft_consensus.cc:2468] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:15.439214 891 raft_consensus.cc:3060] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.439270 2919 consensus_queue.cc:1048] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:15.442341 1071 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.442991 1071 ts_tablet_manager.cc:1403] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.217s user 0.025s sys 0.045s
I20260810 07:53:15.443496 1047 tablet_bootstrap.cc:654] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.445142 309 leader_election.cc:304] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.384735 1064 ts_tablet_manager.cc:1434] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.191s user 0.008s sys 0.001s
I20260810 07:53:15.380461 896 raft_consensus.cc:2468] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:15.435191 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.446681 893 raft_consensus.cc:3060] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.447196 1064 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:15.448904 2426 raft_consensus.cc:2804] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.449294 1071 raft_consensus.cc:359] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:15.449673 308 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.449959 1071 raft_consensus.cc:385] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.450219 1071 raft_consensus.cc:740] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.449306 2426 raft_consensus.cc:493] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.450831 2426 raft_consensus.cc:3060] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.453857 893 raft_consensus.cc:2468] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:15.450870 1071 consensus_queue.cc:260] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.455574 309 leader_election.cc:304] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.457039 1071 ts_tablet_manager.cc:1434] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260810 07:53:15.457803 2911 raft_consensus.cc:2804] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.458256 2911 raft_consensus.cc:697] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:15.459007 2911 consensus_queue.cc:237] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.472754 2397 ts_tablet_manager.cc:1950] T 45c5e309389d402092f59c801923854a P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:15.475270 666 raft_consensus.cc:2468] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
I20260810 07:53:15.476564 308 leader_election.cc:304] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:15.478001 2632 raft_consensus.cc:2804] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:53:15.478820 2632 raft_consensus.cc:697] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
W20260810 07:53:15.479976 755 tablet.cc:2406] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:53:15.479831 2632 consensus_queue.cc:237] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:15.486281 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 47256.4us
W20260810 07:53:15.489718 792 leader_election.cc:343] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.490224 792 leader_election.cc:304] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.491022 2919 raft_consensus.cc:2749] T 6a97561b5ff5490890e756ca5ff50d38 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.491719 2741 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 3.5->[3.6-3.394] Dedup: 3.6->[3.7-3.394]
I20260810 07:53:15.460623 891 raft_consensus.cc:2468] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:15.495049 2934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:15.497114 1047 tablet_bootstrap.cc:492] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:15.497552 1047 ts_tablet_manager.cc:1403] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.132s user 0.009s sys 0.000s
I20260810 07:53:15.500209 1047 raft_consensus.cc:359] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.500860 1047 raft_consensus.cc:385] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.501129 1047 raft_consensus.cc:740] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.501175 309 leader_election.cc:304] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.501891 2459 consensus_queue.cc:1048] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.503271 2459 raft_consensus.cc:2804] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.503721 2459 raft_consensus.cc:697] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:15.504562 2459 consensus_queue.cc:237] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.513559 1071 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.517292 32599 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 4 FOLLOWER]: Rejecting Update request from peer d86331b1fae14fb4b49b99f676a8490a for earlier term 3. Current term is 4. Ops: [3.400-3.400]
I20260810 07:53:15.523100 2950 consensus_queue.cc:1059] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 }, Status: INVALID_TERM, Last received: 3.399, Next index: 400, Last known committed idx: 398, Time since last communication: 0.001s
I20260810 07:53:15.523922 2740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 399 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:53:15.524137 2883 raft_consensus.cc:3055] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 LEADER]: Stepping down as leader of term 3
I20260810 07:53:15.524502 2883 raft_consensus.cc:740] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d86331b1fae14fb4b49b99f676a8490a, State: Running, Role: LEADER
I20260810 07:53:15.525638 2968 raft_consensus.cc:493] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.525472 2883 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 399, Committed index: 399, Last appended: 3.401, Last appended by leader: 401, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:15.526101 2968 raft_consensus.cc:515] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.526734 2883 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 3 FOLLOWER]: Advancing to term 4
I20260810 07:53:15.528659 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 292) took 4659 ms. Trace:
I20260810 07:53:15.529698 1071 tablet_bootstrap.cc:654] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.529814 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:15.530368 410 raft_consensus.cc:2468] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:15.531466 2968 leader_election.cc:290] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:15.529018 2397 rpcz_store.cc:276] 0810 07:53:10.868887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.885124 (+ 16237us) service_pool.cc:224] Handling call
0810 07:53:15.144254 (+4259130us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:15.144295 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.389661 (+245366us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.431493 (+ 41832us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.528600 (+ 97107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":164708,"tablet-delete.queue_time_us":4251367}
I20260810 07:53:15.532485 573 leader_election.cc:304] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:15.533733 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 45c5e309389d402092f59c801923854a (table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]) successfully deleted
I20260810 07:53:15.536002 2968 raft_consensus.cc:2804] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:15.536186 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 291) took 4667 ms. Trace:
I20260810 07:53:15.536388 2968 raft_consensus.cc:493] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:15.536484 2397 rpcz_store.cc:276] 0810 07:53:10.868545 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.892355 (+ 23810us) service_pool.cc:224] Handling call
0810 07:53:15.535904 (+4643549us) ts_tablet_manager.cc:1151] Deleting tablet 54a3bd8369634e4bbfdf8920454357bc
0810 07:53:15.535957 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.536132 (+ 175us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4642823}
I20260810 07:53:15.536892 2968 raft_consensus.cc:3060] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:15.537741 32450 catalog_manager.cc:5017] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 54a3bd8369634e4bbfdf8920454357bc because the tablet was not found. No further retry: Not found: Tablet not found: 54a3bd8369634e4bbfdf8920454357bc
I20260810 07:53:15.538877 2397 tablet_replica.cc:333] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:15.539626 2397 raft_consensus.cc:2243] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:15.540131 2397 raft_consensus.cc:2272] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:15.541006 1047 consensus_queue.cc:260] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
W20260810 07:53:15.545080 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 55384.5us
W20260810 07:53:15.545856 792 leader_election.cc:343] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.546283 792 leader_election.cc:304] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.547049 2919 raft_consensus.cc:2749] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.547895 1064 tablet_bootstrap.cc:654] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:15.548372 792 leader_election.cc:343] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.549247 2397 ts_tablet_manager.cc:1916] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:15.553176 2426 raft_consensus.cc:515] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.554302 1047 ts_tablet_manager.cc:1434] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.056s user 0.007s sys 0.000s
I20260810 07:53:15.554466 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 4 candidate_status { last_received { term: 3 index: 399 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:53:15.555367 1047 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:15.555256 2934 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:15.556660 32484 raft_consensus.cc:2410] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c43f834fbcb447587880136f69c7993 for term 4 because replica has last-logged OpId of term: 3 index: 401, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 399.
I20260810 07:53:15.559955 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:15.566677 2934 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260810 07:53:15.575371 792 leader_election.cc:304] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.569836 1071 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.576305 1071 ts_tablet_manager.cc:1403] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.008s
I20260810 07:53:15.578982 1071 raft_consensus.cc:359] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.579627 1071 raft_consensus.cc:385] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.579905 1071 raft_consensus.cc:740] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.580567 1071 consensus_queue.cc:260] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.593149 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:15.594676 308 leader_election.cc:343] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.595587 2426 leader_election.cc:290] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:15.597335 2968 raft_consensus.cc:515] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.598403 1071 ts_tablet_manager.cc:1434] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260810 07:53:15.601790 1047 tablet_bootstrap.cc:654] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.602962 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.604954 1071 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.604871 2417 ts_tablet_manager.cc:1929] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:15.605273 2417 log.cc:1199] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/8779af4159eb48da894e6a92c2c4a6f2
I20260810 07:53:15.606029 2417 ts_tablet_manager.cc:1950] T 8779af4159eb48da894e6a92c2c4a6f2 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:15.611984 2417 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 354) took 1924 ms. Trace:
I20260810 07:53:15.602648 2919 raft_consensus.cc:2749] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.615826 893 raft_consensus.cc:3060] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.619642 2417 rpcz_store.cc:276] 0810 07:53:13.687556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.687668 (+ 112us) service_pool.cc:224] Handling call
0810 07:53:15.262057 (+1574389us) ts_tablet_manager.cc:1151] Deleting tablet 8779af4159eb48da894e6a92c2c4a6f2
0810 07:53:15.262094 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.431485 (+169391us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.574661 (+143176us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.611914 (+ 37253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1573807}
I20260810 07:53:15.624403 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]) successfully deleted
I20260810 07:53:15.625258 2632 consensus_queue.cc:1048] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.629585 1071 tablet_bootstrap.cc:654] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.632108 893 raft_consensus.cc:2468] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:15.636824 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:15.639518 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:15.640112 410 raft_consensus.cc:3060] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.653678 410 raft_consensus.cc:2468] T a3642f29b38241be9db3cf37ccde2039 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:15.658582 742 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
W20260810 07:53:15.634899 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 86585us
I20260810 07:53:15.665477 573 leader_election.cc:304] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:15.666647 2953 raft_consensus.cc:2804] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.667186 2953 raft_consensus.cc:697] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:15.668143 2953 consensus_queue.cc:237] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.672863 1047 tablet_bootstrap.cc:492] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:15.673460 1047 ts_tablet_manager.cc:1403] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.118s user 0.011s sys 0.000s
I20260810 07:53:15.680843 2968 leader_election.cc:290] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:15.676662 1047 raft_consensus.cc:359] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.688941 1047 raft_consensus.cc:385] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.689311 1047 raft_consensus.cc:740] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.690249 1047 consensus_queue.cc:260] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.692655 1047 ts_tablet_manager.cc:1434] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260810 07:53:15.663267 2397 ts_tablet_manager.cc:1929] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:15.694427 2397 log.cc:1199] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/7eeabaae541548afab4a3ff5ab388035
I20260810 07:53:15.696141 1047 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:15.698521 2397 ts_tablet_manager.cc:1950] T 7eeabaae541548afab4a3ff5ab388035 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:53:15.699885 792 leader_election.cc:343] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.700356 792 leader_election.cc:304] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.701395 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 290) took 4833 ms. Trace:
I20260810 07:53:15.703130 2397 rpcz_store.cc:276] 0810 07:53:10.868244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.889994 (+ 21750us) service_pool.cc:224] Handling call
0810 07:53:15.538728 (+4648734us) ts_tablet_manager.cc:1151] Deleting tablet 7eeabaae541548afab4a3ff5ab388035
0810 07:53:15.538770 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.608989 (+ 70219us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.663128 (+ 54139us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.701334 (+ 38206us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4622243}
I20260810 07:53:15.707319 2919 raft_consensus.cc:2749] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.695176 1071 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.708055 1071 ts_tablet_manager.cc:1403] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.109s user 0.009s sys 0.001s
W20260810 07:53:15.708768 792 leader_election.cc:343] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:15.708973 576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:15.709499 792 leader_election.cc:304] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.710237 2919 raft_consensus.cc:2749] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.710774 1071 raft_consensus.cc:359] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.711479 1071 raft_consensus.cc:385] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.711735 1071 raft_consensus.cc:740] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:15.712345 792 leader_election.cc:343] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.713316 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 7eeabaae541548afab4a3ff5ab388035 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]) successfully deleted
I20260810 07:53:15.713658 2397 tablet_replica.cc:333] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:15.714574 2397 raft_consensus.cc:2243] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:15.715991 2397 pending_rounds.cc:70] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Trying to abort 1 pending ops.
I20260810 07:53:15.716295 2397 pending_rounds.cc:77] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316883013039378432 op_type: NO_OP noop_request { }
I20260810 07:53:15.716807 2397 raft_consensus.cc:2889] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260810 07:53:15.712423 1071 consensus_queue.cc:260] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.720932 2397 raft_consensus.cc:2272] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:15.722862 1071 ts_tablet_manager.cc:1434] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260810 07:53:15.723788 1071 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.723668 309 leader_election.cc:304] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.724701 2426 raft_consensus.cc:2804] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:15.725134 2426 raft_consensus.cc:697] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:15.726218 2426 consensus_queue.cc:237] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.734292 2397 ts_tablet_manager.cc:1916] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:15.742301 792 leader_election.cc:304] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.743225 2919 raft_consensus.cc:2749] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:15.754335 792 leader_election.cc:343] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.754868 792 leader_election.cc:304] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.755620 2919 raft_consensus.cc:2749] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.766067 1071 tablet_bootstrap.cc:654] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:15.765858 792 leader_election.cc:343] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.780535 792 leader_election.cc:304] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:15.782704 2919 raft_consensus.cc:2749] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:15.786198 2919 consensus_queue.cc:1048] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.791168 1064 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:15.791659 1064 ts_tablet_manager.cc:1403] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.345s user 0.008s sys 0.009s
I20260810 07:53:15.794298 1064 raft_consensus.cc:359] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.798420 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:15.800241 1047 tablet_bootstrap.cc:654] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.800942 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.803462 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:15.807734 574 leader_election.cc:343] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:15.809257 2968 consensus_queue.cc:1048] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.809446 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:15.811249 1064 raft_consensus.cc:385] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.811545 1064 raft_consensus.cc:740] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.812191 1064 consensus_queue.cc:260] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.814810 1071 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:15.815290 1071 ts_tablet_manager.cc:1403] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.092s user 0.009s sys 0.000s
I20260810 07:53:15.823251 1047 tablet_bootstrap.cc:492] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:15.823828 1047 ts_tablet_manager.cc:1403] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.128s user 0.014s sys 0.000s
I20260810 07:53:15.826902 1047 raft_consensus.cc:359] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.836407 1071 raft_consensus.cc:359] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.837138 1071 raft_consensus.cc:385] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.837410 1071 raft_consensus.cc:740] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.837298 1047 raft_consensus.cc:385] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.844954 1047 raft_consensus.cc:740] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.845990 1047 consensus_queue.cc:260] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:15.847224 1064 ts_tablet_manager.cc:1434] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.055s user 0.006s sys 0.000s
I20260810 07:53:15.847581 2919 raft_consensus.cc:493] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:15.848074 2919 raft_consensus.cc:515] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.850057 2919 leader_election.cc:290] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:15.853813 1064 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:15.854812 1047 ts_tablet_manager.cc:1434] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
W20260810 07:53:15.858913 967 tablet.cc:2406] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:53:15.849210 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.859968 666 raft_consensus.cc:1240] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Rejecting Update request from peer 9e545e0597e24d61b7265abed2ccdcfb for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260810 07:53:15.838055 1071 consensus_queue.cc:260] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.848102 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:15.857591 414 raft_consensus.cc:1240] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 LEADER]: Rejecting Update request from peer 9e545e0597e24d61b7265abed2ccdcfb for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260810 07:53:15.874436 1071 ts_tablet_manager.cc:1434] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.059s user 0.007s sys 0.000s
I20260810 07:53:15.875615 1071 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:15.887233 2397 ts_tablet_manager.cc:1929] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:15.889983 1047 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:15.890916 2397 log.cc:1199] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/d301af14299040528cf3a033ff5177f8
I20260810 07:53:15.891721 2397 ts_tablet_manager.cc:1950] T d301af14299040528cf3a033ff5177f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:15.899955 2426 consensus_queue.cc:1048] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.900339 1047 tablet_bootstrap.cc:654] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.906255 1064 tablet_bootstrap.cc:654] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:15.929755 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 286) took 5062 ms. Trace:
I20260810 07:53:15.930269 2397 rpcz_store.cc:276] 0810 07:53:10.866951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.917445 (+ 50494us) service_pool.cc:224] Handling call
0810 07:53:15.713484 (+4796039us) ts_tablet_manager.cc:1151] Deleting tablet d301af14299040528cf3a033ff5177f8
0810 07:53:15.713535 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.782811 (+ 69276us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.887099 (+104288us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.929681 (+ 42582us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4764302}
I20260810 07:53:15.933838 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet d301af14299040528cf3a033ff5177f8 (table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]) successfully deleted
I20260810 07:53:15.938746 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:15.943595 2397 tablet_replica.cc:333] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:15.946108 2397 raft_consensus.cc:2243] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:15.947024 2397 raft_consensus.cc:2272] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:15.949158 2397 ts_tablet_manager.cc:1916] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:15.908149 893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:15.955379 1047 tablet_bootstrap.cc:492] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:15.955279 893 raft_consensus.cc:2468] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:15.909277 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3642f29b38241be9db3cf37ccde2039" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:15.955833 1047 ts_tablet_manager.cc:1403] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.002s
I20260810 07:53:15.956090 896 raft_consensus.cc:3060] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:15.959390 1047 raft_consensus.cc:359] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:15.960096 1047 raft_consensus.cc:385] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:15.960357 1047 raft_consensus.cc:740] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:15.949049 2426 consensus_queue.cc:1048] T 01d24209ef6b4c919ce12a702a318577 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:15.950906 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:15.963709 896 raft_consensus.cc:2468] T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:15.973110 1047 consensus_queue.cc:260] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:15.985338 574 consensus_peers.cc:597] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:15.991264 1071 tablet_bootstrap.cc:654] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:15.988693 792 leader_election.cc:343] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.992251 1047 ts_tablet_manager.cc:1434] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.036s user 0.004s sys 0.004s
W20260810 07:53:15.992422 792 leader_election.cc:343] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:15.993525 1047 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:15.994041 2397 ts_tablet_manager.cc:1929] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260810 07:53:15.998126 2397 log.cc:1199] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/ef32f7951f8640e9a13b069816b1cdd5
I20260810 07:53:15.999053 2397 ts_tablet_manager.cc:1950] T ef32f7951f8640e9a13b069816b1cdd5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
W20260810 07:53:16.038822 792 leader_election.cc:343] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.047164 1047 tablet_bootstrap.cc:654] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.049234 2953 raft_consensus.cc:493] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.063766 2953 raft_consensus.cc:515] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:16.067905 2953 leader_election.cc:290] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.069641 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:16.097620 414 raft_consensus.cc:3060] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.102059 2953 raft_consensus.cc:493] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.102735 2953 raft_consensus.cc:515] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:16.105794 2953 leader_election.cc:290] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.106305 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:16.142387 1047 tablet_bootstrap.cc:492] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.142990 1047 ts_tablet_manager.cc:1403] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.150s user 0.006s sys 0.007s
I20260810 07:53:16.143186 2632 raft_consensus.cc:493] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.143952 2632 raft_consensus.cc:515] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.147665 2632 leader_election.cc:290] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.148706 3015 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.146981 1047 raft_consensus.cc:359] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.150131 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 296) took 4917 ms. Trace:
I20260810 07:53:16.151311 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:16.150575 1047 raft_consensus.cc:385] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.150542 2397 rpcz_store.cc:276] 0810 07:53:11.233079 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.233233 (+ 154us) service_pool.cc:224] Handling call
0810 07:53:15.936049 (+4702816us) ts_tablet_manager.cc:1151] Deleting tablet ef32f7951f8640e9a13b069816b1cdd5
0810 07:53:15.943434 (+ 7385us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:15.981467 (+ 38033us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:15.993903 (+ 12436us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:16.150063 (+156160us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4698524}
I20260810 07:53:16.152060 1047 raft_consensus.cc:740] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.153100 2397 tablet_replica.cc:333] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:16.154045 2397 raft_consensus.cc:2243] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:16.154544 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]) successfully deleted
I20260810 07:53:16.154990 2397 raft_consensus.cc:2272] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:16.151280 3015 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:16.157222 2397 ts_tablet_manager.cc:1916] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:16.157202 32450 catalog_manager.cc:5123] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): alter failed for tablet ef32f7951f8640e9a13b069816b1cdd5 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]),no further retry: Not found: Tablet not found: ef32f7951f8640e9a13b069816b1cdd5
I20260810 07:53:16.160495 2740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 5 candidate_status { last_received { term: 3 index: 401 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:16.156523 1047 consensus_queue.cc:260] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.165330 666 raft_consensus.cc:2468] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.168123 1047 ts_tablet_manager.cc:1434] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260810 07:53:16.171278 3015 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:16.173573 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 5 candidate_status { last_received { term: 3 index: 401 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:16.174276 32599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86331b1fae14fb4b49b99f676a8490a in term 4.
I20260810 07:53:16.175596 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters: 72fde54f23f34224ac756f11924c9531
I20260810 07:53:16.176816 3015 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260810 07:53:16.177377 3015 raft_consensus.cc:493] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.178539 1047 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.178928 3015 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 4 FOLLOWER]: Advancing to term 5
I20260810 07:53:16.180508 2426 raft_consensus.cc:493] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:16.181088 2426 raft_consensus.cc:515] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:16.184934 2426 leader_election.cc:290] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:16.189038 1071 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.190726 1071 ts_tablet_manager.cc:1403] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.315s user 0.010s sys 0.004s
I20260810 07:53:16.189915 308 leader_election.cc:304] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.193745 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:16.198413 1047 tablet_bootstrap.cc:654] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.198465 1064 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:16.198964 1064 ts_tablet_manager.cc:1403] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.345s user 0.010s sys 0.007s
I20260810 07:53:16.200973 3015 raft_consensus.cc:515] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:16.201508 1064 raft_consensus.cc:359] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.202251 1064 raft_consensus.cc:385] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.202569 2397 ts_tablet_manager.cc:1929] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:16.202582 1064 raft_consensus.cc:740] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.203001 2397 log.cc:1199] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/0098aaa7be644c9885167dc1ee4929b5
I20260810 07:53:16.203766 2397 ts_tablet_manager.cc:1950] T 0098aaa7be644c9885167dc1ee4929b5 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:16.203615 2740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 5 candidate_status { last_received { term: 3 index: 401 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:53:16.204560 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 298) took 4715 ms. Trace:
I20260810 07:53:16.204922 2397 rpcz_store.cc:276] 0810 07:53:11.489065 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.492835 (+ 3770us) service_pool.cc:224] Handling call
0810 07:53:16.152933 (+4660098us) ts_tablet_manager.cc:1151] Deleting tablet 0098aaa7be644c9885167dc1ee4929b5
0810 07:53:16.152981 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:16.182633 (+ 29652us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:16.202439 (+ 19806us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:16.204509 (+ 2070us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4659516}
I20260810 07:53:16.206358 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 305) took 3912 ms. Trace:
I20260810 07:53:16.206739 2397 rpcz_store.cc:276] 0810 07:53:12.293849 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.294007 (+ 158us) service_pool.cc:224] Handling call
0810 07:53:16.205998 (+3911991us) ts_tablet_manager.cc:1151] Deleting tablet 2a8f390b0db64e1d96c3d8686274cbbf
0810 07:53:16.206043 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:16.206305 (+ 262us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3883716}
I20260810 07:53:16.208078 2397 tablet_replica.cc:333] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:16.208890 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d86331b1fae14fb4b49b99f676a8490a" candidate_term: 5 candidate_status { last_received { term: 3 index: 401 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993"
I20260810 07:53:16.205902 2911 raft_consensus.cc:2804] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.209187 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 0098aaa7be644c9885167dc1ee4929b5 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]) successfully deleted
I20260810 07:53:16.209395 32599 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 4 FOLLOWER]: Advancing to term 5
I20260810 07:53:16.209434 2911 raft_consensus.cc:493] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.209743 2911 raft_consensus.cc:3060] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:16.209910 32450 catalog_manager.cc:5030] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): delete failed for tablet 2a8f390b0db64e1d96c3d8686274cbbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a8f390b0db64e1d96c3d8686274cbbf already in progress: creating tablet
I20260810 07:53:16.208837 3015 leader_election.cc:290] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 5 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:16.208299 1071 raft_consensus.cc:359] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.215917 1071 raft_consensus.cc:385] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.216216 1047 tablet_bootstrap.cc:492] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.216243 1071 raft_consensus.cc:740] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.216693 1047 ts_tablet_manager.cc:1403] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.010s
I20260810 07:53:16.216912 1071 consensus_queue.cc:260] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.219357 1047 raft_consensus.cc:359] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.219817 32599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86331b1fae14fb4b49b99f676a8490a in term 5.
I20260810 07:53:16.220033 1047 raft_consensus.cc:385] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.220299 1047 raft_consensus.cc:740] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.221117 32455 leader_election.cc:304] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters: 72fde54f23f34224ac756f11924c9531
I20260810 07:53:16.222162 1071 ts_tablet_manager.cc:1434] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260810 07:53:16.222445 3015 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 FOLLOWER]: Leader election won for term 5
I20260810 07:53:16.223064 1071 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:16.224958 2911 raft_consensus.cc:515] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.227034 2911 leader_election.cc:290] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.228127 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:16.228885 666 raft_consensus.cc:3060] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.232432 1071 tablet_bootstrap.cc:654] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:16.232857 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 184632us
W20260810 07:53:16.233639 792 leader_election.cc:343] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.234977 2397 raft_consensus.cc:2243] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:16.235630 2397 raft_consensus.cc:2272] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:16.237387 666 raft_consensus.cc:2468] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.238613 308 leader_election.cc:304] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.239015 2919 consensus_queue.cc:1059] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260810 07:53:16.239404 1047 consensus_queue.cc:260] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.239619 2911 raft_consensus.cc:2804] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:16.240159 2397 ts_tablet_manager.cc:1916] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:16.240538 2911 raft_consensus.cc:697] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:16.241379 2911 consensus_queue.cc:237] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:16.246171 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:16.247820 1047 ts_tablet_manager.cc:1434] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.031s user 0.001s sys 0.004s
I20260810 07:53:16.249079 1047 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.256611 1047 tablet_bootstrap.cc:654] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.272775 1047 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.276612 1047 ts_tablet_manager.cc:1403] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.000s
I20260810 07:53:16.284073 1047 raft_consensus.cc:359] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.287653 1047 raft_consensus.cc:385] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.288386 1047 raft_consensus.cc:740] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.203423 1064 consensus_queue.cc:260] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.292606 2968 consensus_queue.cc:1048] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.293503 2426 raft_consensus.cc:493] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.293937 2426 raft_consensus.cc:515] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.295902 2426 leader_election.cc:290] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.298044 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:16.298650 667 raft_consensus.cc:2468] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.300379 308 leader_election.cc:304] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.301509 2426 raft_consensus.cc:2804] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.301988 2426 raft_consensus.cc:493] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.302474 2426 raft_consensus.cc:3060] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.305756 1071 tablet_bootstrap.cc:492] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.306340 1071 ts_tablet_manager.cc:1403] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.083s user 0.012s sys 0.000s
I20260810 07:53:16.310366 1047 consensus_queue.cc:260] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.310914 667 raft_consensus.cc:3060] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.312820 3015 raft_consensus.cc:697] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 LEADER]: Becoming Leader. State: Replica: d86331b1fae14fb4b49b99f676a8490a, State: Running, Role: LEADER
I20260810 07:53:16.313717 3015 consensus_queue.cc:237] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 399, Committed index: 399, Last appended: 3.401, Last appended by leader: 401, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
W20260810 07:53:16.291103 3003 log.cc:927] Time spent T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260810 07:53:16.317755 1047 ts_tablet_manager.cc:1434] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260810 07:53:16.325595 3025 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:16.328493 1047 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.329260 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.329862 896 raft_consensus.cc:2468] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.329617 2426 raft_consensus.cc:515] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.327612 3025 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
I20260810 07:53:16.332181 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:16.332391 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:16.332712 892 raft_consensus.cc:3060] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.333067 2968 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.333534 2968 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:16.335291 1047 tablet_bootstrap.cc:654] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.336084 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98dec9b072ac49959181003cb78567af" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.337572 2968 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.333060 666 raft_consensus.cc:3060] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.338084 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:16.309234 1071 raft_consensus.cc:359] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.341511 1071 raft_consensus.cc:385] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.341823 1071 raft_consensus.cc:740] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.342686 1071 consensus_queue.cc:260] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.343581 892 raft_consensus.cc:2468] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.348932 2911 consensus_queue.cc:1048] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.354471 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.355249 891 raft_consensus.cc:2468] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.370036 1047 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.370620 1047 ts_tablet_manager.cc:1403] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.042s user 0.015s sys 0.000s
I20260810 07:53:16.371327 3013 raft_consensus.cc:3055] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Stepping down as leader of term 1
I20260810 07:53:16.371774 3013 raft_consensus.cc:740] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:16.372653 3013 consensus_queue.cc:260] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:16.373400 2397 ts_tablet_manager.cc:1929] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:16.374028 2397 log.cc:1199] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/8779af4159eb48da894e6a92c2c4a6f2
I20260810 07:53:16.374130 3013 raft_consensus.cc:3060] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:16.374773 2397 ts_tablet_manager.cc:1950] T 8779af4159eb48da894e6a92c2c4a6f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:16.376855 2397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 310) took 2716 ms. Trace:
I20260810 07:53:16.377159 2397 rpcz_store.cc:276] 0810 07:53:13.660209 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.660326 (+ 117us) service_pool.cc:224] Handling call
0810 07:53:16.207821 (+2547495us) ts_tablet_manager.cc:1151] Deleting tablet 8779af4159eb48da894e6a92c2c4a6f2
0810 07:53:16.207966 (+ 145us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:16.334229 (+126263us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:16.373247 (+ 39018us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:16.376794 (+ 3547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13596,"tablet-delete.queue_time_us":2546743}
I20260810 07:53:16.378413 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet 8779af4159eb48da894e6a92c2c4a6f2 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]) successfully deleted
I20260810 07:53:16.384022 1064 ts_tablet_manager.cc:1434] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.185s user 0.005s sys 0.000s
I20260810 07:53:16.385196 1071 ts_tablet_manager.cc:1434] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.079s user 0.007s sys 0.000s
I20260810 07:53:16.386173 1071 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:16.386691 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 152402us
I20260810 07:53:16.389997 2426 leader_election.cc:290] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:16.390992 792 leader_election.cc:343] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.391376 792 leader_election.cc:304] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:16.391270 1047 raft_consensus.cc:359] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.391935 1047 raft_consensus.cc:385] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.392009 2919 raft_consensus.cc:2749] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:16.392217 1047 raft_consensus.cc:740] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.393462 1047 consensus_queue.cc:260] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.394142 309 leader_election.cc:304] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:16.395159 2426 raft_consensus.cc:2749] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:16.397774 1047 ts_tablet_manager.cc:1434] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260810 07:53:16.406313 1071 tablet_bootstrap.cc:654] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.407424 666 raft_consensus.cc:2468] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.405483 1064 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:16.423280 792 leader_election.cc:343] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.434293 308 leader_election.cc:304] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.435235 2426 raft_consensus.cc:2804] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:16.435698 2426 raft_consensus.cc:697] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:16.436575 2426 consensus_queue.cc:237] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.437741 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6c33f86da645e9bf50e45bc229d0a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:16.438519 891 raft_consensus.cc:3060] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:16.441666 501 tablet.cc:2406] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:53:16.445425 1047 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.449375 3041 raft_consensus.cc:493] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.450186 3041 raft_consensus.cc:515] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:16.453080 793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:16.456075 3041 leader_election.cc:290] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.458663 891 raft_consensus.cc:2468] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.457712 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6800738970d47dd8e7cba68945042aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:16.466871 3041 consensus_queue.cc:1048] T 5d7024a39aa040c2b00196f668b776e8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.467684 1071 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.468107 1071 ts_tablet_manager.cc:1403] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.082s user 0.012s sys 0.000s
I20260810 07:53:16.470467 1071 raft_consensus.cc:359] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.471082 1071 raft_consensus.cc:385] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.471331 1071 raft_consensus.cc:740] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.477692 1071 consensus_queue.cc:260] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.470561 667 raft_consensus.cc:2468] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.486214 308 leader_election.cc:304] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.487058 3041 raft_consensus.cc:2804] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.487438 3041 raft_consensus.cc:493] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.487838 3041 raft_consensus.cc:3060] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.490753 1047 tablet_bootstrap.cc:654] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.493012 2426 raft_consensus.cc:493] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.493625 2426 raft_consensus.cc:515] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.497216 2426 leader_election.cc:290] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.487814 1071 ts_tablet_manager.cc:1434] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260810 07:53:16.498114 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:16.501829 308 leader_election.cc:343] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.501969 3041 raft_consensus.cc:515] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:16.504577 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9b4339a26ad540f4b385f69fd0dcdc5d consensus state has no leader
I20260810 07:53:16.504858 1071 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:16.469354 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6800738970d47dd8e7cba68945042aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.472651 792 leader_election.cc:304] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:16.506558 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 83324.8us
I20260810 07:53:16.507426 2919 raft_consensus.cc:2749] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:16.513254 889 raft_consensus.cc:2468] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:16.516264 792 leader_election.cc:343] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.520892 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6800738970d47dd8e7cba68945042aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:16.521637 666 raft_consensus.cc:3060] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.521857 792 leader_election.cc:304] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:16.520032 889 raft_consensus.cc:1275] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Refusing update from remote peer 60781ef3c8b84a8788f43d77e1bb3b19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260810 07:53:16.522564 889 pending_rounds.cc:85] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Aborting all ops after (but not including) 0
I20260810 07:53:16.522954 889 pending_rounds.cc:107] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260810 07:53:16.527101 3041 leader_election.cc:290] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.528015 889 raft_consensus.cc:2889] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260810 07:53:16.531076 1071 tablet_bootstrap.cc:654] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.544457 32412 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: OK, instance_id { permanent_uuid: "72fde54f23f34224ac756f11924c9531" instance_seqno: 1786348392689688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:53:16.521123 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.542520 2919 raft_consensus.cc:2749] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:16.545672 32412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:16.546612 891 raft_consensus.cc:2468] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.550482 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6800738970d47dd8e7cba68945042aa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:16.551169 894 raft_consensus.cc:3060] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.552233 667 raft_consensus.cc:3060] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.561883 894 raft_consensus.cc:2468] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.563773 3041 consensus_queue.cc:1048] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:16.576265 666 raft_consensus.cc:2468] T e6800738970d47dd8e7cba68945042aa P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.578377 309 leader_election.cc:304] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:16.579623 3041 raft_consensus.cc:2804] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.580212 3041 raft_consensus.cc:493] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.580672 3041 raft_consensus.cc:3060] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:16.543892 792 leader_election.cc:343] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.587466 792 leader_election.cc:304] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:16.588828 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 44966.9us
I20260810 07:53:16.589412 1071 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.589856 1071 ts_tablet_manager.cc:1403] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.091s user 0.008s sys 0.004s
I20260810 07:53:16.592283 1071 raft_consensus.cc:359] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.592988 1071 raft_consensus.cc:385] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.593246 1071 raft_consensus.cc:740] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.592274 309 leader_election.cc:304] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:16.594481 2426 raft_consensus.cc:2804] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:16.595098 2426 raft_consensus.cc:697] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:16.595918 2426 consensus_queue.cc:237] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.495087 1064 tablet_bootstrap.cc:654] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.599079 1071 consensus_queue.cc:260] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.601405 1071 ts_tablet_manager.cc:1434] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260810 07:53:16.594970 3013 raft_consensus.cc:2749] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:16.602267 1071 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:16.615760 2911 consensus_queue.cc:1048] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:16.626878 1071 tablet_bootstrap.cc:654] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:16.632920 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:16.641144 1071 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.641671 1071 ts_tablet_manager.cc:1403] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.011s
I20260810 07:53:16.631636 1047 tablet_bootstrap.cc:492] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.655205 1047 ts_tablet_manager.cc:1403] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.210s user 0.011s sys 0.005s
I20260810 07:53:16.654806 1071 raft_consensus.cc:359] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.657409 1071 raft_consensus.cc:385] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.658133 1071 raft_consensus.cc:740] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.659498 1071 consensus_queue.cc:260] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.667734 1047 raft_consensus.cc:359] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.668385 1047 raft_consensus.cc:385] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:53:16.681536 2982 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 2810545162168467244) tablet 00000000000000000000000000000000 cur holder 0x7b5800012c00
I20260810 07:53:16.682626 1071 ts_tablet_manager.cc:1434] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.041s user 0.005s sys 0.002s
I20260810 07:53:16.669628 1047 raft_consensus.cc:740] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.687085 2426 raft_consensus.cc:493] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.687988 2426 raft_consensus.cc:515] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.683732 1047 consensus_queue.cc:260] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.691663 2911 consensus_queue.cc:1048] T d3077b3fe04e406a89560229b76d8242 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.692065 2426 leader_election.cc:290] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.688753 1071 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:16.693841 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:16.695547 308 leader_election.cc:343] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:16.701210 1047 ts_tablet_manager.cc:1434] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.046s user 0.007s sys 0.001s
I20260810 07:53:16.702176 1047 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.715307 666 raft_consensus.cc:1275] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Refusing update from remote peer 60781ef3c8b84a8788f43d77e1bb3b19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260810 07:53:16.718315 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.719070 892 raft_consensus.cc:2468] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:16.729171 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "834a48ebb87a4b999722e05af1feab21" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.730228 1047 tablet_bootstrap.cc:654] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.730227 889 raft_consensus.cc:2393] T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:16.731155 2426 consensus_queue.cc:1048] T 55aaf296555b43f9a27f2926054c8d4b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.741570 1071 tablet_bootstrap.cc:654] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.748585 309 leader_election.cc:304] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:16.751189 2632 raft_consensus.cc:2804] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.751719 2632 raft_consensus.cc:493] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.752127 2632 raft_consensus.cc:3060] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.770947 1071 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.771430 1071 ts_tablet_manager.cc:1403] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.083s user 0.009s sys 0.000s
I20260810 07:53:16.775259 1071 raft_consensus.cc:359] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.775905 1071 raft_consensus.cc:385] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.776162 1071 raft_consensus.cc:740] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.779955 32599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Refusing update from remote peer d86331b1fae14fb4b49b99f676a8490a: Log matching property violated. Preceding OpId in replica: term: 3 index: 399. Preceding OpId from leader: term: 5 index: 402. (index mismatch)
I20260810 07:53:16.780606 1047 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.781785 3025 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 402, Last known committed idx: 398, Time since last communication: 0.000s
I20260810 07:53:16.783628 1047 ts_tablet_manager.cc:1403] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.082s user 0.011s sys 0.000s
I20260810 07:53:16.785851 3065 raft_consensus.cc:493] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.786430 1047 raft_consensus.cc:359] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.787061 1047 raft_consensus.cc:385] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.787307 1047 raft_consensus.cc:740] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.786600 3065 raft_consensus.cc:515] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.791069 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:16.791787 414 raft_consensus.cc:2468] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:16.791913 1047 consensus_queue.cc:260] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.792990 1071 consensus_queue.cc:260] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.793327 573 leader_election.cc:304] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.794283 2968 raft_consensus.cc:2804] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:16.794606 2968 raft_consensus.cc:493] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:16.794914 2968 raft_consensus.cc:3060] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.795078 1071 ts_tablet_manager.cc:1434] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260810 07:53:16.795114 3065 leader_election.cc:290] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.796190 1071 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:16.795848 3013 raft_consensus.cc:493] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:16.796789 3013 raft_consensus.cc:515] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.801401 3013 leader_election.cc:290] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:16.802956 2968 raft_consensus.cc:515] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.811509 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:16.812073 414 raft_consensus.cc:3060] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.819212 414 raft_consensus.cc:2468] T 9f6e61aa223048f8bdc9eee07fa2066c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:16.820309 1047 ts_tablet_manager.cc:1434] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260810 07:53:16.740350 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3077b3fe04e406a89560229b76d8242" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.820451 667 raft_consensus.cc:3060] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.826870 573 leader_election.cc:304] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:16.829386 2968 leader_election.cc:290] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.830529 1071 tablet_bootstrap.cc:654] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.829782 1047 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.831077 3064 raft_consensus.cc:2804] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:16.849189 3062 consensus_queue.cc:1048] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:16.851711 3064 raft_consensus.cc:697] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:16.852689 3064 consensus_queue.cc:237] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.859411 1064 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:16.859922 1064 ts_tablet_manager.cc:1403] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.455s user 0.006s sys 0.007s
I20260810 07:53:16.862910 2911 consensus_queue.cc:1048] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:16.886814 1071 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:16.887362 1071 ts_tablet_manager.cc:1403] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.092s user 0.013s sys 0.001s
I20260810 07:53:16.889851 3041 raft_consensus.cc:515] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.890239 1071 raft_consensus.cc:359] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.890909 1071 raft_consensus.cc:385] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.891249 1071 raft_consensus.cc:740] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.892567 3041 leader_election.cc:290] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.893177 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:16.893524 1047 tablet_bootstrap.cc:654] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.893770 666 raft_consensus.cc:3060] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.912214 1047 tablet_bootstrap.cc:492] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:16.912739 1047 ts_tablet_manager.cc:1403] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.083s user 0.009s sys 0.000s
I20260810 07:53:16.914573 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:16.915181 896 raft_consensus.cc:3060] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:16.920737 3081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:16.921486 3081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:16.925735 1047 raft_consensus.cc:359] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.926484 1047 raft_consensus.cc:385] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.926728 1047 raft_consensus.cc:740] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.927404 1047 consensus_queue.cc:260] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.929207 666 raft_consensus.cc:2468] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.931733 1047 ts_tablet_manager.cc:1434] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260810 07:53:16.932662 1047 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:16.947932 896 raft_consensus.cc:2468] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:16.953680 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d7024a39aa040c2b00196f668b776e8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.957859 1047 tablet_bootstrap.cc:654] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.970136 1064 raft_consensus.cc:359] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.971048 1064 raft_consensus.cc:385] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:16.971379 1064 raft_consensus.cc:740] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:16.993584 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:16.994385 894 raft_consensus.cc:3060] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.001127 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6e61aa223048f8bdc9eee07fa2066c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:16.897025 1071 consensus_queue.cc:260] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.016938 1071 ts_tablet_manager.cc:1434] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.129s user 0.007s sys 0.001s
I20260810 07:53:17.017946 1071 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:17.027894 1071 tablet_bootstrap.cc:654] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:16.777143 2632 raft_consensus.cc:515] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:16.930403 308 leader_election.cc:304] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:17.045100 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 114764us
I20260810 07:53:17.046317 3078 raft_consensus.cc:2804] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:53:17.046715 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 55572.8us
I20260810 07:53:17.047891 2911 raft_consensus.cc:493] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.048656 2911 raft_consensus.cc:515] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.050186 574 leader_election.cc:304] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:17.052476 3064 raft_consensus.cc:2749] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:17.052893 2911 leader_election.cc:290] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.061002 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.062875 2632 leader_election.cc:290] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.070698 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:17.074232 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:17.075322 3064 raft_consensus.cc:493] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.075660 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:17.076144 3064 raft_consensus.cc:515] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.081812 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c95cd95c034d9a9205221d07ae8615" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:17.082536 414 raft_consensus.cc:2468] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:17.084405 573 leader_election.cc:304] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:17.085928 3078 raft_consensus.cc:697] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:17.087563 3078 consensus_queue.cc:237] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.088964 2968 raft_consensus.cc:2804] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:17.089417 2968 raft_consensus.cc:493] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:17.089928 2968 raft_consensus.cc:3060] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.090075 3064 leader_election.cc:290] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:16.976235 1064 consensus_queue.cc:260] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.099571 1047 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:17.100136 1047 ts_tablet_manager.cc:1403] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.168s user 0.016s sys 0.004s
I20260810 07:53:17.106711 1047 raft_consensus.cc:359] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.107504 1047 raft_consensus.cc:385] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.107802 1047 raft_consensus.cc:740] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.108497 1047 consensus_queue.cc:260] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.111285 2968 raft_consensus.cc:515] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.114017 1064 ts_tablet_manager.cc:1434] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.254s user 0.002s sys 0.008s
I20260810 07:53:17.117111 1047 ts_tablet_manager.cc:1434] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.017s user 0.003s sys 0.003s
I20260810 07:53:17.118645 3113 consensus_queue.cc:1048] T 834a48ebb87a4b999722e05af1feab21 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.122350 574 leader_election.cc:304] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:17.123504 3064 raft_consensus.cc:2749] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:17.142541 3065 consensus_queue.cc:1048] T 6a97561b5ff5490890e756ca5ff50d38 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:17.143864 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 70019.3us
W20260810 07:53:17.144551 308 leader_election.cc:343] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:17.145272 308 leader_election.cc:343] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.160594 894 raft_consensus.cc:2468] T 9f6e61aa223048f8bdc9eee07fa2066c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:17.143517 1071 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:17.173637 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 331) took 4595 ms. Trace:
I20260810 07:53:17.183380 574 leader_election.cc:304] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:17.174031 32464 rpcz_store.cc:276] 0810 07:53:12.578321 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.980572 (+1402251us) service_pool.cc:224] Handling call
0810 07:53:14.532322 (+551750us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:53:14.534218 (+ 1896us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:53:17.172889 (+2638671us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:53:17.173267 (+ 378us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:53:17.173566 (+ 299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:14.949134 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:14.949635 (+ 501us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:14.949663 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:14.949678 (+ 15us) tablet.cc:662] Decoding operations
0810 07:53:14.949998 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:14.950042 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:14.950069 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:14.950300 (+ 231us) tablet.cc:701] Row locks acquired
0810 07:53:14.950313 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:14.950465 (+ 152us) write_op.cc:270] Start()
0810 07:53:14.950561 (+ 96us) write_op.cc:276] Timestamp: P: 1786348394950400 usec, L: 0
0810 07:53:14.950575 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:14.951149 (+ 574us) log.cc:844] Serialized 433 byte log entry
0810 07:53:17.102554 (+2151405us) op_driver.cc:464] REPLICATION: finished
0810 07:53:17.170093 (+ 67539us) write_op.cc:301] APPLY: starting
0810 07:53:17.170168 (+ 75us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:17.170299 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:17.170487 (+ 188us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:17.171201 (+ 714us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:17.171516 (+ 315us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:53:17.171597 (+ 81us) write_op.cc:312] APPLY: finished
0810 07:53:17.171900 (+ 303us) log.cc:844] Serialized 29 byte log entry
0810 07:53:17.172145 (+ 245us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:17.172265 (+ 120us) write_op.cc:454] Released schema lock
0810 07:53:17.172316 (+ 51us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":67321,"apply.run_cpu_time_us":2534,"apply.run_wall_time_us":2534,"num_ops":2,"prepare.queue_time_us":315052,"prepare.run_cpu_time_us":2234,"prepare.run_wall_time_us":2233,"replication_time_us":2151825,"threads_started":1}]]}
W20260810 07:53:17.190635 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 48) took 2458 ms (client timeout 1000 ms). Trace:
W20260810 07:53:17.191035 32464 rpcz_store.cc:269] 0810 07:53:14.732100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.190355 (+2458255us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:17.190465 (+ 110us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:17.192519 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 37) took 2271 ms (client timeout 1000 ms). Trace:
W20260810 07:53:17.193018 32464 rpcz_store.cc:269] 0810 07:53:14.921096 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.192337 (+2271241us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:17.192444 (+ 107us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:17.194485 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 35) took 1338 ms (client timeout 1000 ms). Trace:
I20260810 07:53:17.196259 3064 raft_consensus.cc:2749] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:17.197788 32526 master_runner.cc:187] Error getting master registration for 127.31.139.126:38193: OK, instance_id { permanent_uuid: "72fde54f23f34224ac756f11924c9531" instance_seqno: 1786348392689688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:53:17.199558 32526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:53:17.201395 3064 consensus_queue.cc:1048] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.209676 3081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:17.210616 3081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:17.165273 1071 ts_tablet_manager.cc:1403] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.148s user 0.009s sys 0.003s
I20260810 07:53:17.202049 3025 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:17.224978 3025 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
W20260810 07:53:17.224869 32464 rpcz_store.cc:269] 0810 07:53:15.855679 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.194285 (+1338606us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:17.194414 (+ 129us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:17.227288 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 49) took 1338 ms (client timeout 1000 ms). Trace:
W20260810 07:53:17.227691 32464 rpcz_store.cc:269] 0810 07:53:15.889234 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.227127 (+1337893us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:17.227231 (+ 104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:17.228942 32464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 38) took 1153 ms (client timeout 1000 ms). Trace:
W20260810 07:53:17.229308 32464 rpcz_store.cc:269] 0810 07:53:16.075899 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.228795 (+1152896us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:17.228892 (+ 97us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:17.231575 32464 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 332) took 4089 ms. Trace:
W20260810 07:53:17.232638 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 114789us
I20260810 07:53:17.233098 2968 leader_election.cc:290] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:17.234236 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 66568.3us
I20260810 07:53:17.162063 3013 raft_consensus.cc:493] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.235173 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c95cd95c034d9a9205221d07ae8615" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:17.235148 3013 raft_consensus.cc:515] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.235838 414 raft_consensus.cc:3060] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.238037 3013 leader_election.cc:290] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:17.115505 1064 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:17.202286 792 leader_election.cc:343] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.242554 32464 rpcz_store.cc:276] 0810 07:53:13.142553 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.230511 (+4087958us) service_pool.cc:224] Handling call
0810 07:53:17.231520 (+ 1009us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:17.247218 3065 raft_consensus.cc:493] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:53:17.247509 980 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.31.139.125:46845)
I20260810 07:53:17.247805 3065 raft_consensus.cc:515] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.252444 3105 raft_consensus.cc:493] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.252677 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:17.253036 3105 raft_consensus.cc:515] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.255259 3105 leader_election.cc:290] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.256667 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.257272 666 raft_consensus.cc:2468] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:17.257454 32464 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:43806 (request call id 333) took 3859 ms. Trace:
I20260810 07:53:17.257746 32464 rpcz_store.cc:276] 0810 07:53:13.398665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.256952 (+3858287us) service_pool.cc:224] Handling call
0810 07:53:17.257388 (+ 436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:17.258699 308 leader_election.cc:304] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:17.258636 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:17.260160 573 leader_election.cc:343] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.260291 3065 leader_election.cc:290] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.262050 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:17.262633 888 raft_consensus.cc:3060] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.265141 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.265731 886 raft_consensus.cc:2468] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:17.266491 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.267098 891 raft_consensus.cc:2468] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:17.268905 309 leader_election.cc:304] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.270643 1064 tablet_bootstrap.cc:654] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.271443 3113 raft_consensus.cc:2804] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:17.271781 3113 raft_consensus.cc:493] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:17.272087 3113 raft_consensus.cc:3060] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.276007 414 raft_consensus.cc:2468] T 92c95cd95c034d9a9205221d07ae8615 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:17.276247 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 334) took 3877 ms. Trace:
I20260810 07:53:17.279472 32464 rpcz_store.cc:276] 0810 07:53:13.399126 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.270410 (+3871284us) service_pool.cc:224] Handling call
0810 07:53:17.276177 (+ 5767us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:17.281351 980 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.139.125:46845)
I20260810 07:53:17.282410 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 335) took 3563 ms. Trace:
I20260810 07:53:17.283917 32464 rpcz_store.cc:276] 0810 07:53:13.718783 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.281890 (+3563107us) service_pool.cc:224] Handling call
0810 07:53:17.282350 (+ 460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:17.290338 573 leader_election.cc:304] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:17.286280 2632 raft_consensus.cc:2804] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:17.294106 2632 raft_consensus.cc:493] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:17.295039 2632 raft_consensus.cc:3060] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.298916 888 raft_consensus.cc:2468] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:17.302314 2968 raft_consensus.cc:2804] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:17.307081 3105 consensus_queue.cc:1048] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.307968 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.308511 676 raft_consensus.cc:2468] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:17.308609 3105 consensus_queue.cc:1048] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:17.224092 1071 raft_consensus.cc:359] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.310228 1071 raft_consensus.cc:385] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.310525 1071 raft_consensus.cc:740] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.311138 2968 raft_consensus.cc:697] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:17.311139 1071 consensus_queue.cc:260] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.312525 2968 consensus_queue.cc:237] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.313990 309 leader_election.cc:304] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.315078 3111 raft_consensus.cc:2804] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:17.316008 3111 raft_consensus.cc:697] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:17.317219 3111 consensus_queue.cc:237] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.329128 3113 raft_consensus.cc:515] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.331276 3113 leader_election.cc:290] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.336534 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:17.338070 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 705fc015928140a58db979445068365f from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:17.334944 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:17.336972 1071 ts_tablet_manager.cc:1434] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.117s user 0.008s sys 0.000s
I20260810 07:53:17.339371 3105 consensus_queue.cc:1048] T 999f3e79ee4b4c368b60b202735c2556 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.355350 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:17.356046 888 raft_consensus.cc:3060] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:17.357081 308 leader_election.cc:343] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.358260 1064 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:17.358803 1064 ts_tablet_manager.cc:1403] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.243s user 0.004s sys 0.010s
I20260810 07:53:17.364789 3105 consensus_queue.cc:1048] T 7d6c33f86da645e9bf50e45bc229d0a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.370260 1071 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:17.370559 3013 raft_consensus.cc:493] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.387754 1047 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:17.371153 3013 raft_consensus.cc:515] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.396764 3013 leader_election.cc:290] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:17.397853 3039 op_tracker.cc:284] OpTracker waiting for 1 outstanding ops to complete now for 1.030s
I20260810 07:53:17.398214 3039 op_tracker.cc:286] Dumping up to 50 currently running ops:
I20260810 07:53:17.398470 3039 op_tracker.cc:290] R-P WriteOp [type=LEADER, start_time=2026-08-10 07:53:17, state=WriteOpState 0x7b58000e4900 [op_id=(), ts=7316883035455721472, rows=[MUTATE (int8 entry_type=<redacted>, string entry_id=<redacted>) SET metadata=<redacted>]]]
I20260810 07:53:17.408205 1047 tablet_bootstrap.cc:654] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.399712 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]). Expected version 1 got 0
I20260810 07:53:17.416031 32464 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 336) took 3528 ms. Trace:
I20260810 07:53:17.416400 32464 rpcz_store.cc:276] 0810 07:53:13.887421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.337954 (+3450533us) service_pool.cc:224] Handling call
0810 07:53:17.415755 (+ 77801us) spinlock_profiling.cc:243] Waited 76.3 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ed41 00007f30cf440d37 00007f30cf440cc2 00007f30cf440bf1 00007f30cf440b6c 00007f30cf43fe22 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:17.415964 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175558400}
I20260810 07:53:17.418047 3105 raft_consensus.cc:493] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:53:17.418573 980 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.31.139.125:46845)
I20260810 07:53:17.418514 3105 raft_consensus.cc:515] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.420475 500 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:17.424433 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.425549 3105 leader_election.cc:290] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:17.426048 308 leader_election.cc:343] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.426836 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.427450 896 raft_consensus.cc:2468] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:17.428707 309 leader_election.cc:304] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.430718 3117 raft_consensus.cc:2804] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:17.431063 3117 raft_consensus.cc:493] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:17.431365 3117 raft_consensus.cc:3060] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.416157 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
I20260810 07:53:17.434005 2632 raft_consensus.cc:515] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.436368 2632 leader_election.cc:290] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.438467 3111 raft_consensus.cc:493] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.438405 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:17.438990 676 raft_consensus.cc:3060] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.438877 3111 raft_consensus.cc:515] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.441275 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.445828 3111 leader_election.cc:290] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.446152 32464 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:43806 (request call id 337) took 3284 ms. Trace:
I20260810 07:53:17.446467 32464 rpcz_store.cc:276] 0810 07:53:14.161338 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.445636 (+3284298us) service_pool.cc:224] Handling call
0810 07:53:17.446087 (+ 451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:17.447182 32464 master_service.cc:960] ReplaceTablet: received request to replace tablet 5c1ddea2d9f3462092fcbaee7774cd01 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:17.447683 32464 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:43806 (request call id 338) took 2196 ms. Trace:
I20260810 07:53:17.447957 32464 rpcz_store.cc:276] 0810 07:53:15.251380 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:17.447089 (+2195709us) service_pool.cc:224] Handling call
0810 07:53:17.447636 (+ 547us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:17.449160 308 leader_election.cc:343] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.449174 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c95cd95c034d9a9205221d07ae8615" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.455263 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:17.457473 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:17.458109 885 raft_consensus.cc:3060] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.465543 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:17.466542 885 raft_consensus.cc:2468] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:17.467092 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]). Expected version 1 got 0
I20260810 07:53:17.470794 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c95cd95c034d9a9205221d07ae8615" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:17.471669 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.473083 1064 raft_consensus.cc:359] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.474309 1047 tablet_bootstrap.cc:492] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:17.474709 1047 ts_tablet_manager.cc:1403] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.000s
W20260810 07:53:17.475018 792 leader_election.cc:343] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.475169 1064 raft_consensus.cc:385] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.475463 792 leader_election.cc:304] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.475613 1064 raft_consensus.cc:740] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.476380 1064 consensus_queue.cc:260] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.477909 3013 raft_consensus.cc:2749] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:17.479713 1064 ts_tablet_manager.cc:1434] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.121s user 0.006s sys 0.000s
I20260810 07:53:17.486302 1047 raft_consensus.cc:359] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.487007 1047 raft_consensus.cc:385] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.487314 1047 raft_consensus.cc:740] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.488047 1047 consensus_queue.cc:260] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.492403 1064 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:17.500908 1047 ts_tablet_manager.cc:1434] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.026s user 0.006s sys 0.003s
I20260810 07:53:17.502082 1047 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:17.540568 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1fe71e69dfd74a3b826050bd7edc6b6a consensus state has no leader
I20260810 07:53:17.541817 676 raft_consensus.cc:2468] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:17.543417 308 leader_election.cc:304] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:17.541813 1047 tablet_bootstrap.cc:654] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.539906 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:17.554643 1071 tablet_bootstrap.cc:654] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:17.557430 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260810 07:53:17.559960 309 leader_election.cc:343] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.560451 309 leader_election.cc:304] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.563715 3113 raft_consensus.cc:2804] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:17.564541 3113 raft_consensus.cc:697] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:17.565768 3113 consensus_queue.cc:237] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.567049 3041 consensus_queue.cc:1048] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:17.571837 2911 raft_consensus.cc:2749] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:17.527833 3117 raft_consensus.cc:515] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.596155 3117 leader_election.cc:290] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.595934 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:17.598009 308 leader_election.cc:343] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.630129 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:17.630736 894 raft_consensus.cc:3060] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.636503 3153 tablet.cc:1724] T 8718f998623e4f1c8b7b8be815e184a3 P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:17.645501 1071 tablet_bootstrap.cc:492] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:17.646111 1071 ts_tablet_manager.cc:1403] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.276s user 0.013s sys 0.000s
I20260810 07:53:17.647547 3158 tablet.cc:1724] T 8718f998623e4f1c8b7b8be815e184a3 P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260810 07:53:17.682065 2982 lock_manager.cc:585] Waited 2 seconds to obtain row lock on key '<redacted>' (key hash 2810545162168467244) tablet 00000000000000000000000000000000 cur holder 0x7b5800012c00
I20260810 07:53:17.699537 1071 raft_consensus.cc:359] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.704833 1071 raft_consensus.cc:385] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.649708 1047 tablet_bootstrap.cc:492] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:17.705485 1047 ts_tablet_manager.cc:1403] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.204s user 0.012s sys 0.004s
W20260810 07:53:17.709323 3137 log.cc:927] Time spent T a3642f29b38241be9db3cf37ccde2039 P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.167s user 0.001s sys 0.000s
I20260810 07:53:17.696691 888 raft_consensus.cc:2468] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:17.715828 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 239303us
I20260810 07:53:17.709096 1047 raft_consensus.cc:359] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.719066 1047 raft_consensus.cc:385] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.719406 1047 raft_consensus.cc:740] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.720260 1047 consensus_queue.cc:260] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.722826 1047 ts_tablet_manager.cc:1434] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
W20260810 07:53:17.723444 792 leader_election.cc:343] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.725370 1047 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:17.736981 1071 raft_consensus.cc:740] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.737675 1064 tablet_bootstrap.cc:654] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.751322 894 raft_consensus.cc:2468] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:17.750941 1071 consensus_queue.cc:260] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.753686 1071 ts_tablet_manager.cc:1434] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
I20260810 07:53:17.759819 309 leader_election.cc:304] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:17.763520 3117 raft_consensus.cc:2804] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:17.764031 3117 raft_consensus.cc:697] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:17.765069 3117 consensus_queue.cc:237] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:17.777082 3149 log.cc:927] Time spent T 834a48ebb87a4b999722e05af1feab21 P 9e545e0597e24d61b7265abed2ccdcfb: Append to log took a long time: real 0.187s user 0.001s sys 0.002s
I20260810 07:53:17.780233 1047 tablet_bootstrap.cc:654] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.786111 1071 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:17.787482 309 leader_election.cc:304] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:17.788115 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 63942.2us
I20260810 07:53:17.792645 3163 tablet.cc:1724] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260810 07:53:17.803556 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:17.804943 3111 raft_consensus.cc:2804] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:17.805496 3111 raft_consensus.cc:697] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:17.806435 3111 consensus_queue.cc:237] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:17.818841 574 leader_election.cc:343] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.821637 1047 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:17.822072 1047 ts_tablet_manager.cc:1403] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.097s user 0.010s sys 0.000s
W20260810 07:53:17.824091 574 leader_election.cc:343] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.824087 2632 consensus_queue.cc:1048] T e6800738970d47dd8e7cba68945042aa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:17.825619 574 leader_election.cc:343] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.826118 574 leader_election.cc:304] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:17.827131 3065 raft_consensus.cc:2749] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:17.825358 1047 raft_consensus.cc:359] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.836771 1047 raft_consensus.cc:385] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.837163 1047 raft_consensus.cc:740] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.838050 1047 consensus_queue.cc:260] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.840807 1047 ts_tablet_manager.cc:1434] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260810 07:53:17.843685 1071 tablet_bootstrap.cc:654] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.897519 3062 consensus_queue.cc:1048] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:17.899461 1071 tablet_bootstrap.cc:492] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:17.899891 1071 ts_tablet_manager.cc:1403] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.114s user 0.007s sys 0.003s
I20260810 07:53:17.902588 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:17.834658 1064 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:17.915251 1064 ts_tablet_manager.cc:1403] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.423s user 0.011s sys 0.008s
I20260810 07:53:17.918076 1064 raft_consensus.cc:359] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.918802 1064 raft_consensus.cc:385] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.919104 1064 raft_consensus.cc:740] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.919901 1064 consensus_queue.cc:260] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.921958 1064 ts_tablet_manager.cc:1434] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260810 07:53:17.904773 800 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 77872.9us
I20260810 07:53:17.902372 1071 raft_consensus.cc:359] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.929311 1071 raft_consensus.cc:385] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:17.929592 1071 raft_consensus.cc:740] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:17.930686 1071 consensus_queue.cc:260] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.935945 1071 ts_tablet_manager.cc:1434] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260810 07:53:17.937282 1071 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:17.914160 792 leader_election.cc:304] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:17.945875 3146 raft_consensus.cc:2804] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:17.946337 3146 raft_consensus.cc:493] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:17.946775 3146 raft_consensus.cc:3060] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:17.959543 3111 raft_consensus.cc:493] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:17.960220 3111 raft_consensus.cc:515] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:17.970317 1071 tablet_bootstrap.cc:654] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:17.973963 1047 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:17.980064 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:17.982013 308 leader_election.cc:343] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:17.985198 3111 leader_election.cc:290] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:17.989604 1047 tablet_bootstrap.cc:654] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:18.009785 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 105289us
W20260810 07:53:18.016175 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 62504.7us
W20260810 07:53:18.016887 792 leader_election.cc:343] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.030808 1071 tablet_bootstrap.cc:492] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.031334 1071 ts_tablet_manager.cc:1403] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.094s user 0.014s sys 0.002s
I20260810 07:53:18.034992 1071 raft_consensus.cc:359] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.038175 1071 raft_consensus.cc:385] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.038547 1071 raft_consensus.cc:740] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.039188 1071 consensus_queue.cc:260] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.052045 1071 ts_tablet_manager.cc:1434] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.020s user 0.001s sys 0.004s
I20260810 07:53:18.066838 1047 tablet_bootstrap.cc:492] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:18.067287 1047 ts_tablet_manager.cc:1403] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.094s user 0.011s sys 0.001s
I20260810 07:53:18.067085 3111 consensus_queue.cc:1048] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:18.078586 1071 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.091476 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.093058 894 raft_consensus.cc:2468] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:18.093478 1071 tablet_bootstrap.cc:654] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.106338 1071 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.106805 1071 ts_tablet_manager.cc:1403] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.028s user 0.016s sys 0.004s
I20260810 07:53:18.115787 1071 raft_consensus.cc:359] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.116532 1071 raft_consensus.cc:385] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.120886 1071 raft_consensus.cc:740] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.122237 1071 consensus_queue.cc:260] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:18.124081 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:18.126330 1071 ts_tablet_manager.cc:1434] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260810 07:53:18.130458 309 leader_election.cc:304] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:18.131590 3111 raft_consensus.cc:2804] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:18.132190 3111 raft_consensus.cc:493] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:18.132800 3111 raft_consensus.cc:3060] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.149475 1071 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.161099 1071 tablet_bootstrap.cc:654] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.163851 3065 raft_consensus.cc:493] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.164487 3065 raft_consensus.cc:515] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.176227 1064 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:18.180346 2968 raft_consensus.cc:493] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.180980 2968 raft_consensus.cc:515] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.189531 1071 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.190078 1071 ts_tablet_manager.cc:1403] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.005s
I20260810 07:53:18.193172 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:18.193356 1071 raft_consensus.cc:359] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.193845 414 raft_consensus.cc:2468] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:18.194168 1071 raft_consensus.cc:385] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.194484 1071 raft_consensus.cc:740] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.196194 573 leader_election.cc:304] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.196762 1071 consensus_queue.cc:260] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.200260 3064 raft_consensus.cc:2804] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:18.200673 3064 raft_consensus.cc:493] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:18.201084 3064 raft_consensus.cc:3060] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.212430 1064 tablet_bootstrap.cc:654] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.226052 3065 leader_election.cc:290] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.230764 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:18.229693 1071 ts_tablet_manager.cc:1434] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.039s user 0.009s sys 0.001s
I20260810 07:53:18.241544 1071 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.241852 2968 leader_election.cc:290] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:18.253063 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 122727us
I20260810 07:53:18.264541 1047 raft_consensus.cc:359] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.265422 1047 raft_consensus.cc:385] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.265724 1047 raft_consensus.cc:740] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.266477 1047 consensus_queue.cc:260] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.272444 3062 consensus_queue.cc:1048] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:18.276799 3146 raft_consensus.cc:515] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.279856 3146 leader_election.cc:290] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:18.284590 3065 consensus_queue.cc:1048] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:18.299360 3125 raft_consensus.cc:493] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.300166 3125 raft_consensus.cc:515] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.303452 1071 tablet_bootstrap.cc:654] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.303179 3125 leader_election.cc:290] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:18.308928 3113 consensus_queue.cc:1048] T a86401e669fa4358ae33ce2249b3a98e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:18.317051 3111 raft_consensus.cc:515] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:18.330493 792 leader_election.cc:343] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.331043 792 leader_election.cc:304] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:18.331817 3125 raft_consensus.cc:2749] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:18.333827 3113 raft_consensus.cc:493] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.334507 3113 raft_consensus.cc:515] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.334468 3064 raft_consensus.cc:515] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.338196 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:18.338713 414 raft_consensus.cc:3060] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.339089 1071 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.339517 1071 ts_tablet_manager.cc:1403] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.098s user 0.000s sys 0.014s
I20260810 07:53:18.341861 1071 raft_consensus.cc:359] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.342455 1071 raft_consensus.cc:385] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.342729 1071 raft_consensus.cc:740] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.343318 1071 consensus_queue.cc:260] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.345108 414 raft_consensus.cc:2468] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:18.346539 573 leader_election.cc:304] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.348039 3062 raft_consensus.cc:2804] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:18.349037 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:18.349457 1071 ts_tablet_manager.cc:1434] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.010s user 0.005s sys 0.002s
I20260810 07:53:18.349934 676 raft_consensus.cc:2468] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:18.350615 3113 leader_election.cc:290] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.351282 308 leader_election.cc:304] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.353096 3113 raft_consensus.cc:2804] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:18.353440 3113 raft_consensus.cc:493] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:18.349025 3062 raft_consensus.cc:697] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:18.353765 3113 raft_consensus.cc:3060] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.354430 3062 consensus_queue.cc:237] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.361253 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:18.361474 3111 leader_election.cc:290] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.361934 676 raft_consensus.cc:3060] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.369926 676 raft_consensus.cc:2468] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:18.390156 308 leader_election.cc:304] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.391114 3117 raft_consensus.cc:2804] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:18.391836 3117 raft_consensus.cc:697] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:18.395147 3117 consensus_queue.cc:237] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.371716 1047 ts_tablet_manager.cc:1434] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.304s user 0.011s sys 0.000s
I20260810 07:53:18.423988 1047 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:18.379216 3064 leader_election.cc:290] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.439304 3117 raft_consensus.cc:493] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.439844 3117 raft_consensus.cc:515] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.442162 3117 leader_election.cc:290] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.443114 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:18.444674 308 leader_election.cc:343] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.444676 3117 raft_consensus.cc:493] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.446547 667 raft_consensus.cc:3060] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.446672 3117 raft_consensus.cc:515] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.449733 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:18.451392 308 leader_election.cc:343] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.452194 3117 leader_election.cc:290] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.458179 3117 consensus_queue.cc:1048] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:18.466903 1071 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.472831 1071 tablet_bootstrap.cc:654] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.484987 3064 raft_consensus.cc:493] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.485455 3064 raft_consensus.cc:515] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.487861 3064 leader_election.cc:290] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.488581 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:18.500903 1071 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.501487 1071 ts_tablet_manager.cc:1403] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.008s
I20260810 07:53:18.502884 3113 raft_consensus.cc:515] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.504051 1071 raft_consensus.cc:359] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.504678 1071 raft_consensus.cc:385] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.509275 1071 raft_consensus.cc:740] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.510368 1071 consensus_queue.cc:260] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.513123 1071 ts_tablet_manager.cc:1434] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
W20260810 07:53:18.532878 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:18.553795 676 raft_consensus.cc:3060] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.568336 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:18.568997 664 raft_consensus.cc:3060] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.573122 1047 tablet_bootstrap.cc:654] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.576071 664 raft_consensus.cc:2468] T 85f7b2b4d5484e08a5e0f56c7e01f07e P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:18.575980 3117 consensus_queue.cc:1048] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:18.577250 308 leader_election.cc:304] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.578528 3078 raft_consensus.cc:2804] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:18.579034 3078 raft_consensus.cc:697] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:18.579937 3078 consensus_queue.cc:237] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.584854 3113 leader_election.cc:290] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.598896 1071 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.623442 1064 tablet_bootstrap.cc:492] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:18.624147 1064 ts_tablet_manager.cc:1403] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.448s user 0.016s sys 0.011s
I20260810 07:53:18.626093 1071 tablet_bootstrap.cc:654] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.642100 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.642832 891 raft_consensus.cc:2468] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:18.646134 1047 tablet_bootstrap.cc:492] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:18.646550 1047 ts_tablet_manager.cc:1403] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.223s user 0.011s sys 0.000s
I20260810 07:53:18.649413 1047 raft_consensus.cc:359] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.650044 1047 raft_consensus.cc:385] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.650290 1047 raft_consensus.cc:740] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.650851 1047 consensus_queue.cc:260] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.658164 1064 raft_consensus.cc:359] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.660995 1047 ts_tablet_manager.cc:1434] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260810 07:53:18.661978 1047 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:18.670138 1047 tablet_bootstrap.cc:654] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:18.682664 2982 lock_manager.cc:585] Waited 3 seconds to obtain row lock on key '<redacted>' (key hash 2810545162168467244) tablet 00000000000000000000000000000000 cur holder 0x7b5800012c00
I20260810 07:53:18.691349 1047 tablet_bootstrap.cc:492] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:18.691766 1047 ts_tablet_manager.cc:1403] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20260810 07:53:18.659812 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bd442cbd9542ab8a521ededc6b1e90" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:18.718688 3078 raft_consensus.cc:493] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.719213 1064 raft_consensus.cc:385] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.719352 891 raft_consensus.cc:3060] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.719529 1064 raft_consensus.cc:740] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.719486 3078 raft_consensus.cc:515] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.720295 1064 consensus_queue.cc:260] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.722419 1064 ts_tablet_manager.cc:1434] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.098s user 0.006s sys 0.000s
I20260810 07:53:18.723342 1064 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:18.725970 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:18.726622 667 raft_consensus.cc:2468] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:18.727622 891 raft_consensus.cc:2468] T 23bd442cbd9542ab8a521ededc6b1e90 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:18.728252 308 leader_election.cc:304] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:18.668005 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 300511us
I20260810 07:53:18.694518 1047 raft_consensus.cc:359] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.733412 1047 raft_consensus.cc:385] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.729688 3078 leader_election.cc:290] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.733731 1047 raft_consensus.cc:740] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.744215 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.734418 1047 consensus_queue.cc:260] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.744786 894 raft_consensus.cc:2468] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:18.746109 3216 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:18.746685 3216 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:18.747943 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.748821 889 raft_consensus.cc:2393] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:18.751847 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 403 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:18.752954 3078 raft_consensus.cc:2804] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:18.753317 3078 raft_consensus.cc:493] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:18.753655 3078 raft_consensus.cc:3060] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.754137 889 raft_consensus.cc:3060] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.754575 1047 ts_tablet_manager.cc:1434] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.063s user 0.006s sys 0.001s
I20260810 07:53:18.767021 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.771287 3216 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:18.772292 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 403 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:18.773936 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:18.780458 1071 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:18.784157 1071 ts_tablet_manager.cc:1403] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.185s user 0.014s sys 0.003s
I20260810 07:53:18.787623 1071 raft_consensus.cc:359] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.788290 1071 raft_consensus.cc:385] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.789135 3062 raft_consensus.cc:493] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.789649 3062 raft_consensus.cc:515] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.788568 1071 raft_consensus.cc:740] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:18.791770 3062 leader_election.cc:290] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.792042 1071 consensus_queue.cc:260] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.796598 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:18.797199 895 raft_consensus.cc:3060] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.799279 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a61dbafb4f7444e690cea0d003d38f22" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.800060 877 raft_consensus.cc:2393] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:18.803107 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:18.804713 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 76726.8us
I20260810 07:53:18.807988 1071 ts_tablet_manager.cc:1434] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.023s user 0.004s sys 0.003s
I20260810 07:53:18.816176 1071 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:18.821892 3216 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:18.826977 3217 consensus_queue.cc:1048] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:18.838439 895 raft_consensus.cc:2468] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:18.841215 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:18.841761 886 raft_consensus.cc:3060] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.849603 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:18.850397 676 raft_consensus.cc:3060] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.855965 3217 raft_consensus.cc:493] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.856464 3217 raft_consensus.cc:515] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.858001 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:18.859056 3217 leader_election.cc:290] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.859295 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.860092 890 raft_consensus.cc:2393] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:18.843819 3078 raft_consensus.cc:515] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.870193 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:18.871889 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfc73a83a0f4048ad9dded62d074e19" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.872619 892 raft_consensus.cc:2393] T ebfc73a83a0f4048ad9dded62d074e19 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:18.873121 414 raft_consensus.cc:2393] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:18.876668 1047 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:18.878686 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c97418290e04f60884c721ded14495a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:18.879441 414 raft_consensus.cc:2393] T 3c97418290e04f60884c721ded14495a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:18.879521 3219 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.880002 3219 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.881956 3219 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.883415 1047 tablet_bootstrap.cc:654] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.882076 3078 leader_election.cc:290] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.904923 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:18.907895 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:18.909092 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:18.910682 3062 raft_consensus.cc:493] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:18.911190 3062 raft_consensus.cc:515] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:18.912545 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 23bd442cbd9542ab8a521ededc6b1e90 consensus state has no leader
I20260810 07:53:18.913408 3062 leader_election.cc:290] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:18.916488 574 leader_election.cc:304] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:18.917598 3062 raft_consensus.cc:2749] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:18.919435 676 raft_consensus.cc:2468] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:18.923206 574 leader_election.cc:304] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:18.923938 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:18.924364 3064 raft_consensus.cc:2749] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:18.924525 414 raft_consensus.cc:2468] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:18.925395 1047 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:18.925905 573 leader_election.cc:304] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:18.926787 3062 raft_consensus.cc:2804] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:18.927147 3062 raft_consensus.cc:493] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:18.927518 3062 raft_consensus.cc:3060] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:18.925907 1047 ts_tablet_manager.cc:1403] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.010s
I20260810 07:53:18.933226 1047 raft_consensus.cc:359] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:18.934018 1047 raft_consensus.cc:385] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:18.934372 1047 raft_consensus.cc:740] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:18.935105 309 leader_election.cc:343] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.935575 309 leader_election.cc:304] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:18.936764 3217 raft_consensus.cc:2749] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:18.937498 3217 consensus_queue.cc:1048] T 1e629e89eb734ade82b3dcbf11c971f8 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:18.939559 3219 consensus_queue.cc:1048] T 83b68614cd8444a08c7820a33c59442d P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:18.947809 308 leader_election.cc:343] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.957057 1047 consensus_queue.cc:260] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:18.948783 308 leader_election.cc:343] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:18.967329 886 raft_consensus.cc:2468] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:18.967814 1047 ts_tablet_manager.cc:1434] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260810 07:53:18.961404 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:18.968811 1047 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:18.968827 895 raft_consensus.cc:2468] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:18.978703 1064 tablet_bootstrap.cc:654] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:18.957878 742 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:18.999830 676 raft_consensus.cc:3060] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.007359 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.009153 3078 consensus_queue.cc:1048] T a61dbafb4f7444e690cea0d003d38f22 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:18.894538 1071 tablet_bootstrap.cc:654] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.019321 1047 tablet_bootstrap.cc:654] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.022395 574 leader_election.cc:304] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:19.023844 3188 consensus_queue.cc:1048] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:19.034896 792 leader_election.cc:304] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.038357 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.039703 3217 raft_consensus.cc:493] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:19.040220 3217 raft_consensus.cc:515] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.042966 3217 leader_election.cc:290] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.043184 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:19.044601 308 leader_election.cc:343] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:19.045367 1064 tablet_bootstrap.cc:492] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:19.045804 1064 ts_tablet_manager.cc:1403] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.323s user 0.011s sys 0.000s
I20260810 07:53:19.049170 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:19.049958 889 raft_consensus.cc:2393] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:18.960348 3062 raft_consensus.cc:515] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.079536 309 leader_election.cc:304] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.080957 3217 raft_consensus.cc:2804] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:19.081426 3217 raft_consensus.cc:493] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:19.081830 3217 raft_consensus.cc:3060] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.090687 3217 raft_consensus.cc:515] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.093672 3217 leader_election.cc:290] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.093562 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:19.094424 667 raft_consensus.cc:2393] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
W20260810 07:53:19.095079 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:19.095360 3062 leader_election.cc:290] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.096777 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:19.097512 414 raft_consensus.cc:2393] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:19.116245 3188 raft_consensus.cc:2749] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:19.125502 3199 raft_consensus.cc:2749] T 3c97418290e04f60884c721ded14495a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:19.136456 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 112976us
I20260810 07:53:19.149391 1064 raft_consensus.cc:359] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.150295 1064 raft_consensus.cc:385] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.150633 1064 raft_consensus.cc:740] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.152978 1064 consensus_queue.cc:260] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.161438 1071 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:19.161908 1071 ts_tablet_manager.cc:1403] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.346s user 0.007s sys 0.004s
W20260810 07:53:19.163204 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 128369us
I20260810 07:53:19.171510 1047 tablet_bootstrap.cc:492] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:19.172014 1047 ts_tablet_manager.cc:1403] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.203s user 0.023s sys 0.010s
I20260810 07:53:19.177286 1047 raft_consensus.cc:359] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.177946 1047 raft_consensus.cc:385] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.178965 792 leader_election.cc:304] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:19.179826 3199 raft_consensus.cc:2804] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:19.180580 3199 raft_consensus.cc:697] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:19.180920 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.181496 895 raft_consensus.cc:2468] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:19.181550 3199 consensus_queue.cc:237] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.178215 1047 raft_consensus.cc:740] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.192488 1047 consensus_queue.cc:260] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.208006 1071 raft_consensus.cc:359] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.209502 1071 raft_consensus.cc:385] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.209772 1071 raft_consensus.cc:740] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:19.211551 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 132118us
I20260810 07:53:19.212282 1047 ts_tablet_manager.cc:1434] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.040s user 0.007s sys 0.000s
W20260810 07:53:19.213627 309 leader_election.cc:343] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:19.214008 309 leader_election.cc:304] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.214645 3217 raft_consensus.cc:2749] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:19.215602 309 leader_election.cc:343] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:19.216027 309 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.217736 1064 ts_tablet_manager.cc:1434] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.172s user 0.008s sys 0.000s
I20260810 07:53:19.217926 3217 raft_consensus.cc:2749] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:19.219058 309 leader_election.cc:304] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.225811 3217 consensus_queue.cc:1048] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:19.228394 1071 consensus_queue.cc:260] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.230002 1047 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:19.219336 1064 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:19.225117 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46252.1us
I20260810 07:53:19.251008 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.251613 889 raft_consensus.cc:2468] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:19.253535 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:19.254046 889 raft_consensus.cc:3060] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.264523 3064 raft_consensus.cc:493] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:19.265019 3064 raft_consensus.cc:515] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.267055 3064 leader_election.cc:290] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.268757 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:19.269111 889 raft_consensus.cc:2468] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:53:19.269202 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 50215.3us
I20260810 07:53:19.269317 414 raft_consensus.cc:2468] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:19.270365 573 leader_election.cc:304] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:19.271246 3064 raft_consensus.cc:2804] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:19.271672 3064 raft_consensus.cc:493] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:19.272063 3064 raft_consensus.cc:3060] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.273046 3217 raft_consensus.cc:2749] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:19.274291 3078 consensus_queue.cc:1048] T ebfc73a83a0f4048ad9dded62d074e19 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:19.279081 1071 ts_tablet_manager.cc:1434] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.117s user 0.007s sys 0.000s
I20260810 07:53:19.286152 1071 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:19.291322 3064 raft_consensus.cc:515] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.298866 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:19.299430 414 raft_consensus.cc:3060] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.302186 855 tablet_service.cc:1511] Processing CreateTablet for tablet 741404076635419bbbd6ce075fc44ccb (DEFAULT_TABLE table=default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.303411 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741404076635419bbbd6ce075fc44ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.306528 414 raft_consensus.cc:2468] T 1cb66183211f45d4a756f70c5e331f87 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:19.307650 622 tablet_service.cc:1511] Processing CreateTablet for tablet 741404076635419bbbd6ce075fc44ccb (DEFAULT_TABLE table=default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.308629 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741404076635419bbbd6ce075fc44ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.311383 3064 leader_election.cc:290] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.313877 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.314486 888 raft_consensus.cc:2468] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:19.324450 862 tablet_service.cc:1511] Processing CreateTablet for tablet 74d80c441b2b418994b46250a5c7af44 (DEFAULT_TABLE table=default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.325614 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d80c441b2b418994b46250a5c7af44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.324544 385 tablet_service.cc:1511] Processing CreateTablet for tablet 741404076635419bbbd6ce075fc44ccb (DEFAULT_TABLE table=default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.329610 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741404076635419bbbd6ce075fc44ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.333001 369 tablet_service.cc:1511] Processing CreateTablet for tablet 74d80c441b2b418994b46250a5c7af44 (DEFAULT_TABLE table=default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.334137 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d80c441b2b418994b46250a5c7af44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.336463 630 tablet_service.cc:1511] Processing CreateTablet for tablet 74d80c441b2b418994b46250a5c7af44 (DEFAULT_TABLE table=default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.337687 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74d80c441b2b418994b46250a5c7af44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.346400 3078 raft_consensus.cc:493] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:19.346935 3078 raft_consensus.cc:515] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.349581 3078 leader_election.cc:290] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.353158 864 tablet_service.cc:1511] Processing CreateTablet for tablet 2abec82a68a54082afa1d8e72aba6617 (DEFAULT_TABLE table=default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.354326 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abec82a68a54082afa1d8e72aba6617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.369472 1064 tablet_bootstrap.cc:654] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:19.376947 3258 log.cc:927] Time spent T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260810 07:53:19.389057 856 tablet_service.cc:1511] Processing CreateTablet for tablet 5dad896bfa0e4c5cbc4907a75082b064 (DEFAULT_TABLE table=default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.390151 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dad896bfa0e4c5cbc4907a75082b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.400523 1047 tablet_bootstrap.cc:654] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.404832 371 tablet_service.cc:1511] Processing CreateTablet for tablet 2abec82a68a54082afa1d8e72aba6617 (DEFAULT_TABLE table=default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:19.417222 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 107593us
I20260810 07:53:19.418423 573 leader_election.cc:304] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:19.419648 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abec82a68a54082afa1d8e72aba6617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.420603 3064 raft_consensus.cc:2804] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:19.421103 3064 raft_consensus.cc:697] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:19.421875 3064 consensus_queue.cc:237] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.423143 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb66183211f45d4a756f70c5e331f87" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:19.423981 895 raft_consensus.cc:3060] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.428046 309 leader_election.cc:304] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.436164 1064 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:19.436686 1064 ts_tablet_manager.cc:1403] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.218s user 0.010s sys 0.000s
I20260810 07:53:19.447156 1064 raft_consensus.cc:359] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.451071 1047 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:19.451617 1047 ts_tablet_manager.cc:1403] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.222s user 0.011s sys 0.000s
I20260810 07:53:19.453998 622 tablet_service.cc:1511] Processing CreateTablet for tablet 2abec82a68a54082afa1d8e72aba6617 (DEFAULT_TABLE table=default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.454452 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:19.455150 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abec82a68a54082afa1d8e72aba6617. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:19.461324 3244 log.cc:927] Time spent T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Append to log took a long time: real 0.322s user 0.035s sys 0.000s
I20260810 07:53:19.467918 574 leader_election.cc:304] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:19.468766 3188 raft_consensus.cc:2804] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:19.469290 3188 raft_consensus.cc:697] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:19.470203 3188 consensus_queue.cc:237] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.475291 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]). Expected version 1 got 0
W20260810 07:53:19.507241 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 214829us
W20260810 07:53:19.508451 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 90103.2us
I20260810 07:53:19.439448 963 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38193
I20260810 07:53:19.440037 895 raft_consensus.cc:2468] T 1cb66183211f45d4a756f70c5e331f87 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:19.516860 1064 raft_consensus.cc:385] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.517284 1064 raft_consensus.cc:740] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.525079 630 tablet_service.cc:1511] Processing CreateTablet for tablet 5dad896bfa0e4c5cbc4907a75082b064 (DEFAULT_TABLE table=default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.526757 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dad896bfa0e4c5cbc4907a75082b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.527179 1071 tablet_bootstrap.cc:654] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.466535 1047 raft_consensus.cc:359] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.538630 1047 raft_consensus.cc:385] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.539131 1047 raft_consensus.cc:740] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.520452 1064 consensus_queue.cc:260] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.554800 32470 catalog_manager.cc:5719] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:19.552024 852 tablet_service.cc:1511] Processing CreateTablet for tablet 3dfa1b6f604c452e9f346ecc18b5b691 (DEFAULT_TABLE table=default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.575019 1064 ts_tablet_manager.cc:1434] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.130s user 0.003s sys 0.002s
I20260810 07:53:19.576277 1047 consensus_queue.cc:260] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.528172 3219 raft_consensus.cc:2804] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:19.584620 3219 raft_consensus.cc:493] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:19.585059 3219 raft_consensus.cc:3060] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.596591 3219 raft_consensus.cc:515] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.556066 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]). Expected version 1 got 0
W20260810 07:53:19.601819 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 145585us
I20260810 07:53:19.603710 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]). Expected version 3 got 0
I20260810 07:53:19.611356 1047 ts_tablet_manager.cc:1434] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.159s user 0.006s sys 0.000s
I20260810 07:53:19.613529 624 tablet_service.cc:1511] Processing CreateTablet for tablet 3dfa1b6f604c452e9f346ecc18b5b691 (DEFAULT_TABLE table=default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.614858 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfa1b6f604c452e9f346ecc18b5b691. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:19.615479 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:19.617053 3271 consensus_queue.cc:1048] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:19.604367 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]). Expected version 1 got 0
I20260810 07:53:19.620608 32470 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]). Expected version 1 got 0
I20260810 07:53:19.635569 3219 leader_election.cc:290] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:19.636802 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet c9920a33cc93419ca3284a0b009236df
I20260810 07:53:19.638710 32460 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 312) took 8757 ms. Trace:
I20260810 07:53:19.642462 645 tablet_service.cc:1558] Processing DeleteTablet for tablet c9920a33cc93419ca3284a0b009236df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.639053 32460 rpcz_store.cc:276] 0810 07:53:10.881295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.959604 (+ 78309us) service_pool.cc:224] Handling call
0810 07:53:10.968489 (+ 8885us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:10.970205 (+ 1716us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:10.987107 (+ 16902us) catalog_manager.cc:2850] Locking tablets
0810 07:53:11.326264 (+339157us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:11.460126 (+133862us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:11.460342 (+ 216us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:19.636498 (+8176156us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:19.638602 (+ 2104us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:11.328600 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:11.329469 (+ 869us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:11.329515 (+ 46us) write_op.cc:435] Acquired schema lock
0810 07:53:11.329543 (+ 28us) tablet.cc:662] Decoding operations
0810 07:53:11.330132 (+ 589us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:11.330189 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:11.330228 (+ 39us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:11.330663 (+ 435us) tablet.cc:701] Row locks acquired
0810 07:53:11.330681 (+ 18us) write_op.cc:260] PREPARE: finished
0810 07:53:11.330866 (+ 185us) write_op.cc:270] Start()
0810 07:53:11.330999 (+ 133us) write_op.cc:276] Timestamp: P: 1786348391330765 usec, L: 0
0810 07:53:11.331021 (+ 22us) op_driver.cc:348] REPLICATION: starting
0810 07:53:11.393971 (+ 62950us) log.cc:844] Serialized 771 byte log entry
0810 07:53:11.394836 (+ 865us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf964951 00007f30cf9f7208 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1 00007f30cfe8a50d 00000000005bd8c4 00000000005bd6f9
0810 07:53:11.455959 (+ 61123us) op_driver.cc:464] REPLICATION: finished
0810 07:53:11.457980 (+ 2021us) write_op.cc:301] APPLY: starting
0810 07:53:11.458047 (+ 67us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:11.458178 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:11.458200 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:11.458725 (+ 525us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:11.459087 (+ 362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:11.459153 (+ 66us) write_op.cc:312] APPLY: finished
0810 07:53:11.459486 (+ 333us) log.cc:844] Serialized 29 byte log entry
0810 07:53:11.459707 (+ 221us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:11.459788 (+ 81us) write_op.cc:454] Released schema lock
0810 07:53:11.459848 (+ 60us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:11.460732 (+ 884us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:11.460761 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1843,"apply.run_cpu_time_us":2186,"apply.run_wall_time_us":2654,"num_ops":2,"prepare.queue_time_us":156,"prepare.run_cpu_time_us":5762,"prepare.run_wall_time_us":66508,"raft.queue_time_us":18114,"raft.run_cpu_time_us":1392,"raft.run_wall_time_us":1392,"replication_time_us":124815,"spinlock_wait_cycles":143339520}]]}
W20260810 07:53:19.644914 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 108908us
I20260810 07:53:19.646337 32461 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da] alter complete (version 1)
I20260810 07:53:19.647413 32465 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet cec929b5f0e14f6586ed50137ecd741b
I20260810 07:53:19.649171 32465 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 313) took 8752 ms. Trace:
I20260810 07:53:19.649569 32465 rpcz_store.cc:276] 0810 07:53:10.896351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.896510 (+ 159us) service_pool.cc:224] Handling call
0810 07:53:10.897840 (+ 1330us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:10.898423 (+ 583us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:10.898570 (+ 147us) catalog_manager.cc:2850] Locking tablets
0810 07:53:11.336179 (+437609us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:11.460159 (+123980us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:11.460424 (+ 265us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:19.647167 (+8186743us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:19.649098 (+ 1931us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:11.395262 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:11.395819 (+ 557us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:11.395851 (+ 32us) write_op.cc:435] Acquired schema lock
0810 07:53:11.395872 (+ 21us) tablet.cc:662] Decoding operations
0810 07:53:11.396294 (+ 422us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:11.396348 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:11.396380 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:11.397763 (+ 1383us) tablet.cc:701] Row locks acquired
0810 07:53:11.397790 (+ 27us) write_op.cc:260] PREPARE: finished
0810 07:53:11.397949 (+ 159us) write_op.cc:270] Start()
0810 07:53:11.398058 (+ 109us) write_op.cc:276] Timestamp: P: 1786348391397850 usec, L: 0
0810 07:53:11.398083 (+ 25us) op_driver.cc:348] REPLICATION: starting
0810 07:53:11.398805 (+ 722us) log.cc:844] Serialized 731 byte log entry
0810 07:53:11.457695 (+ 58890us) op_driver.cc:464] REPLICATION: finished
0810 07:53:11.458251 (+ 556us) write_op.cc:301] APPLY: starting
0810 07:53:11.458321 (+ 70us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:11.458436 (+ 115us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:11.458687 (+ 251us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:11.459097 (+ 410us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:11.459362 (+ 265us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:11.459441 (+ 79us) write_op.cc:312] APPLY: finished
0810 07:53:11.459687 (+ 246us) log.cc:844] Serialized 29 byte log entry
0810 07:53:11.459817 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:11.459907 (+ 90us) write_op.cc:454] Released schema lock
0810 07:53:11.459969 (+ 62us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":391,"apply.run_cpu_time_us":1994,"apply.run_wall_time_us":2329,"num_ops":2,"prepare.queue_time_us":57150,"prepare.run_cpu_time_us":2844,"prepare.run_wall_time_us":3878,"raft.queue_time_us":15657,"raft.run_cpu_time_us":26,"raft.run_wall_time_us":27,"replication_time_us":59510}]]}
I20260810 07:53:19.654331 639 tablet_service.cc:1558] Processing DeleteTablet for tablet cec929b5f0e14f6586ed50137ecd741b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.658807 1071 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:19.659332 1071 ts_tablet_manager.cc:1403] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.373s user 0.014s sys 0.000s
I20260810 07:53:19.612668 1047 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:19.670796 3188 consensus_queue.cc:1048] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:19.678181 308 leader_election.cc:343] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:19.682320 1071 raft_consensus.cc:359] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.683112 1071 raft_consensus.cc:385] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.683373 1071 raft_consensus.cc:740] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.684069 1071 consensus_queue.cc:260] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.686153 3283 tablet_replica.cc:333] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:19.686349 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2fe3153512b40a99caf3fb9b81fd7a1" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:19.687140 891 raft_consensus.cc:2393] T e2fe3153512b40a99caf3fb9b81fd7a1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:19.687804 3283 raft_consensus.cc:2243] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:19.688371 1071 ts_tablet_manager.cc:1434] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.029s user 0.008s sys 0.000s
I20260810 07:53:19.689637 1071 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:19.700183 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:19.700811 676 raft_consensus.cc:3060] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.708472 676 raft_consensus.cc:2468] T c43aeafc19a240e6991b69249b0516fa P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:19.710062 369 tablet_service.cc:1511] Processing CreateTablet for tablet 3dfa1b6f604c452e9f346ecc18b5b691 (DEFAULT_TABLE table=default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.711170 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfa1b6f604c452e9f346ecc18b5b691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.711949 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:19.712895 3283 raft_consensus.cc:2272] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:19.713524 371 tablet_service.cc:1511] Processing CreateTablet for tablet 5dad896bfa0e4c5cbc4907a75082b064 (DEFAULT_TABLE table=default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.717636 385 tablet_service.cc:1558] Processing DeleteTablet for tablet c9920a33cc93419ca3284a0b009236df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:19.719137 384 tablet_service.cc:1558] Processing DeleteTablet for tablet cec929b5f0e14f6586ed50137ecd741b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:19.723596 1071 tablet_bootstrap.cc:654] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:19.745409 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 74852.4us
I20260810 07:53:19.748324 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43aeafc19a240e6991b69249b0516fa" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:19.748934 892 raft_consensus.cc:3060] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:19.752130 1071 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:19.752599 1071 ts_tablet_manager.cc:1403] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.005s
I20260810 07:53:19.755407 1071 raft_consensus.cc:359] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.756085 1071 raft_consensus.cc:385] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.756395 1071 raft_consensus.cc:740] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.757880 3286 tablet_replica.cc:333] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:19.758567 892 raft_consensus.cc:2468] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:19.759230 371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dad896bfa0e4c5cbc4907a75082b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.771695 1071 consensus_queue.cc:260] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.778606 1071 ts_tablet_manager.cc:1434] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260810 07:53:19.788793 3286 raft_consensus.cc:2243] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:19.789825 3286 raft_consensus.cc:2272] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:19.791159 3283 ts_tablet_manager.cc:1916] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:19.779693 1071 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:19.724443 1047 tablet_bootstrap.cc:654] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.805730 3286 ts_tablet_manager.cc:1916] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:19.821445 3219 consensus_queue.cc:1048] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:19.811180 312 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 92252us
I20260810 07:53:19.825287 860 tablet_service.cc:1558] Processing DeleteTablet for tablet cec929b5f0e14f6586ed50137ecd741b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:19.825292 859 tablet_service.cc:1558] Processing DeleteTablet for tablet c9920a33cc93419ca3284a0b009236df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:19.828162 856 tablet_service.cc:1511] Processing CreateTablet for tablet a7e077b6b69044c2b6d3d2bb7d99ee29 (DEFAULT_TABLE table=default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.828702 1047 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:19.829197 1047 ts_tablet_manager.cc:1403] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.217s user 0.010s sys 0.002s
I20260810 07:53:19.831671 645 tablet_service.cc:1511] Processing CreateTablet for tablet a7e077b6b69044c2b6d3d2bb7d99ee29 (DEFAULT_TABLE table=default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.832732 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e077b6b69044c2b6d3d2bb7d99ee29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.851913 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfa1b6f604c452e9f346ecc18b5b691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.857740 1064 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:19.860954 3297 tablet_replica.cc:333] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:19.861992 3297 raft_consensus.cc:2243] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:19.862592 3297 raft_consensus.cc:2272] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:19.864063 1064 tablet_bootstrap.cc:654] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.865028 3297 ts_tablet_manager.cc:1916] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:19.865770 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e077b6b69044c2b6d3d2bb7d99ee29. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:19.880280 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 49637.1us
I20260810 07:53:19.881026 308 leader_election.cc:304] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:19.882217 3217 raft_consensus.cc:2804] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:19.882726 3217 raft_consensus.cc:697] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:19.883652 3217 consensus_queue.cc:237] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.888626 1071 tablet_bootstrap.cc:654] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:19.899762 1047 raft_consensus.cc:359] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.900547 1047 raft_consensus.cc:385] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.912906 1047 raft_consensus.cc:740] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:19.913723 1047 consensus_queue.cc:260] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:19.916162 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 46589.3us
I20260810 07:53:19.926136 1071 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:19.928925 1071 ts_tablet_manager.cc:1403] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.149s user 0.008s sys 0.003s
I20260810 07:53:19.941464 385 tablet_service.cc:1511] Processing CreateTablet for tablet a7e077b6b69044c2b6d3d2bb7d99ee29 (DEFAULT_TABLE table=default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.942524 385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e077b6b69044c2b6d3d2bb7d99ee29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.945484 384 tablet_service.cc:1511] Processing CreateTablet for tablet 87ae23cb445e43fc9dc2f45097190fec (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.946676 384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ae23cb445e43fc9dc2f45097190fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.954138 309 leader_election.cc:304] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:19.954931 3217 raft_consensus.cc:2749] T e2fe3153512b40a99caf3fb9b81fd7a1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:19.958480 639 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.959653 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.961300 622 tablet_service.cc:1511] Processing CreateTablet for tablet 87ae23cb445e43fc9dc2f45097190fec (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:19.962466 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ae23cb445e43fc9dc2f45097190fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:19.990157 630 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.944404 1071 raft_consensus.cc:359] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:19.992251 1071 raft_consensus.cc:385] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:19.999876 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:19.992594 1071 raft_consensus.cc:740] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.008181 1071 consensus_queue.cc:260] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.010658 1071 ts_tablet_manager.cc:1434] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.070s user 0.005s sys 0.000s
I20260810 07:53:20.011680 1071 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:20.012665 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 34) took 9624 ms (client timeout 1000 ms). Trace:
W20260810 07:53:20.013023 32462 rpcz_store.cc:269] 0810 07:53:10.388465 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:10.670556 (+282091us) service_pool.cc:224] Handling call
0810 07:53:20.012471 (+9341915us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87796,"reported_tablets":35}
W20260810 07:53:20.037017 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 81000.2us
W20260810 07:53:20.058037 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:20.063931 380 tablet_service.cc:1511] Processing CreateTablet for tablet 63812f4779914930b4e22d020a51a33b (DEFAULT_TABLE table=default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.092424 380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63812f4779914930b4e22d020a51a33b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:20.102213 309 leader_election.cc:343] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.102795 309 leader_election.cc:304] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:20.103535 3280 raft_consensus.cc:2749] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:20.104321 3009 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:20.105149 3280 consensus_queue.cc:1048] T 23bd442cbd9542ab8a521ededc6b1e90 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:20.105023 3009 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:20.111562 2927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 3 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:20.125548 1071 tablet_bootstrap.cc:654] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:20.125819 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 174564us
I20260810 07:53:20.116003 2927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:20.139701 864 tablet_service.cc:1511] Processing CreateTablet for tablet 87ae23cb445e43fc9dc2f45097190fec (DEFAULT_TABLE table=default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.154289 852 tablet_service.cc:1511] Processing CreateTablet for tablet 63812f4779914930b4e22d020a51a33b (DEFAULT_TABLE table=default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.159941 1047 ts_tablet_manager.cc:1434] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.330s user 0.007s sys 0.000s
I20260810 07:53:20.161166 1047 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:20.161724 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ae23cb445e43fc9dc2f45097190fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.163784 852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63812f4779914930b4e22d020a51a33b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.171344 373 tablet_service.cc:1511] Processing CreateTablet for tablet cca378598d0c45178e497fd212638d77 (DEFAULT_TABLE table=default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.175870 373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cca378598d0c45178e497fd212638d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.178731 369 tablet_service.cc:1511] Processing CreateTablet for tablet 446ed19d914548ea864a43b2bb8c7f5a (DEFAULT_TABLE table=default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.180052 369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446ed19d914548ea864a43b2bb8c7f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.169847 856 tablet_service.cc:1511] Processing CreateTablet for tablet 446ed19d914548ea864a43b2bb8c7f5a (DEFAULT_TABLE table=default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.183509 1071 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:20.186813 1071 ts_tablet_manager.cc:1403] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.175s user 0.007s sys 0.006s
I20260810 07:53:20.187793 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446ed19d914548ea864a43b2bb8c7f5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:20.196267 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 82611.5us
I20260810 07:53:20.203691 3009 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:20.229754 855 tablet_service.cc:1511] Processing CreateTablet for tablet cca378598d0c45178e497fd212638d77 (DEFAULT_TABLE table=default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.245169 1047 tablet_bootstrap.cc:654] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:20.251076 3269 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.251781 3269 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.255728 3283 ts_tablet_manager.cc:1929] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:20.256201 3283 log.cc:1199] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/c9920a33cc93419ca3284a0b009236df
I20260810 07:53:20.256929 3269 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.261966 3269 raft_consensus.cc:493] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.262430 3269 raft_consensus.cc:515] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.263213 2701 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" instance_seqno: 1786348364618434) as {username='slave'} at 127.31.139.67:50457; Asking this server to re-register.
I20260810 07:53:20.264732 3269 leader_election.cc:290] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.266115 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.267077 645 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.268021 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e0a84839427099a9da06e2b970ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.279385 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.280373 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.281816 645 tablet_service.cc:1511] Processing CreateTablet for tablet 63812f4779914930b4e22d020a51a33b (DEFAULT_TABLE table=default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.282812 645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63812f4779914930b4e22d020a51a33b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.289789 624 tablet_service.cc:1511] Processing CreateTablet for tablet 446ed19d914548ea864a43b2bb8c7f5a (DEFAULT_TABLE table=default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.290874 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446ed19d914548ea864a43b2bb8c7f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.296408 1064 tablet_bootstrap.cc:492] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:20.297016 1064 ts_tablet_manager.cc:1403] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.439s user 0.018s sys 0.060s
I20260810 07:53:20.308779 3280 consensus_queue.cc:1048] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:20.313347 3283 ts_tablet_manager.cc:1950] T c9920a33cc93419ca3284a0b009236df P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:20.318143 630 tablet_service.cc:1511] Processing CreateTablet for tablet cca378598d0c45178e497fd212638d77 (DEFAULT_TABLE table=default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:20.319234 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cca378598d0c45178e497fd212638d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.321496 3219 raft_consensus.cc:493] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.322029 3219 raft_consensus.cc:515] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:20.338011 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 140948us
I20260810 07:53:20.339138 1064 raft_consensus.cc:359] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.340103 1064 raft_consensus.cc:385] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:20.340371 1064 raft_consensus.cc:740] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.340696 3276 consensus_queue.cc:1048] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:20.341030 1064 consensus_queue.cc:260] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.347808 2740 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 3 FOLLOWER]: Advancing to term 5
W20260810 07:53:20.352300 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 225762us
W20260810 07:53:20.352998 980 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.31.139.125:46845)
I20260810 07:53:20.360806 3219 leader_election.cc:290] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.368139 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:20.369170 410 raft_consensus.cc:2468] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:20.370886 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:20.371592 410 raft_consensus.cc:2468] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:20.372678 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet c9920a33cc93419ca3284a0b009236df (table default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b]) successfully deleted
W20260810 07:53:20.388660 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 64165.4us
I20260810 07:53:20.394416 1071 raft_consensus.cc:359] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.395407 1071 raft_consensus.cc:385] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:20.395792 1071 raft_consensus.cc:740] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.396585 1071 consensus_queue.cc:260] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.397590 573 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:20.398885 573 leader_election.cc:304] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:20.398967 3271 raft_consensus.cc:2804] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:20.399608 3188 raft_consensus.cc:2804] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:20.399940 3188 raft_consensus.cc:493] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:20.400276 3188 raft_consensus.cc:3060] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:20.408979 3271 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:20.409626 3271 raft_consensus.cc:3060] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:20.414106 3217 raft_consensus.cc:493] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.414899 3217 raft_consensus.cc:515] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.418311 3217 leader_election.cc:290] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.435139 3188 raft_consensus.cc:515] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.438925 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cca378598d0c45178e497fd212638d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:20.440932 1064 ts_tablet_manager.cc:1434] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.130s user 0.006s sys 0.000s
I20260810 07:53:20.442004 1064 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:20.442600 3199 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.443097 3199 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.445147 3199 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:20.316550 3283 tablet_replica.cc:333] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:20.447427 3283 raft_consensus.cc:2243] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:20.450716 1071 ts_tablet_manager.cc:1434] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.264s user 0.008s sys 0.003s
I20260810 07:53:20.453481 3286 ts_tablet_manager.cc:1929] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:20.454191 3286 log.cc:1199] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/c9920a33cc93419ca3284a0b009236df
I20260810 07:53:20.455687 3286 ts_tablet_manager.cc:1950] T c9920a33cc93419ca3284a0b009236df P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:20.347728 2741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59478 (request call id 0) took 6271 ms. Trace:
I20260810 07:53:20.459005 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:20.480027 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
W20260810 07:53:20.480436 309 consensus_peers.cc:597] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:20.480569 414 raft_consensus.cc:3060] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:20.488436 414 raft_consensus.cc:2468] T 8d717f39a6704199abc34b220d795e48 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:20.489526 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:20.491335 963 heartbeater.cc:461] Registering TS with master...
I20260810 07:53:20.473884 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:20.476850 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:20.498836 3283 raft_consensus.cc:2272] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:20.502027 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:20.504371 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:20.501614 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:20.505007 885 raft_consensus.cc:2468] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:20.511058 32463 catalog_manager.cc:6264] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73] alter complete (version 2)
I20260810 07:53:20.521878 573 leader_election.cc:304] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:20.522790 3064 raft_consensus.cc:2804] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:20.523249 3064 raft_consensus.cc:697] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
W20260810 07:53:20.523842 309 leader_election.cc:343] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.524027 3064 consensus_queue.cc:237] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:20.531463 309 leader_election.cc:343] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.537890 862 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:20.543238 859 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:20.543484 860 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:20.544297 849 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:20.553210 32468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 34) took 8332 ms (client timeout 1000 ms). Trace:
I20260810 07:53:20.555160 849 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:20.555828 862 tablet_service.cc:1558] Processing DeleteTablet for tablet 371fa2534984499b95c0945af358e8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:59 UTC) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:20.559340 32469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.67:54007 (request call id 33) took 9414 ms (client timeout 1000 ms). Trace:
I20260810 07:53:20.560748 862 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:20.561133 3283 ts_tablet_manager.cc:1916] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:20.561357 32469 rpcz_store.cc:269] 0810 07:53:11.144871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.207467 (+ 62596us) service_pool.cc:224] Handling call
0810 07:53:11.390420 (+182953us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:20.559156 (+9168736us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:12.793605 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:12.794037 (+ 432us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:12.794060 (+ 23us) write_op.cc:435] Acquired schema lock
0810 07:53:12.794072 (+ 12us) tablet.cc:662] Decoding operations
0810 07:53:12.794323 (+ 251us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:12.794365 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:12.794389 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:12.794538 (+ 149us) tablet.cc:701] Row locks acquired
0810 07:53:12.794551 (+ 13us) write_op.cc:260] PREPARE: finished
0810 07:53:12.794665 (+ 114us) write_op.cc:270] Start()
0810 07:53:12.794761 (+ 96us) write_op.cc:276] Timestamp: P: 1786348392794540 usec, L: 0
0810 07:53:12.794775 (+ 14us) op_driver.cc:348] REPLICATION: starting
0810 07:53:12.805331 (+ 10556us) log.cc:844] Serialized 533 byte log entry
0810 07:53:12.827180 (+ 21849us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf964951 00007f30cf9f7208 00007f30cfe86f61 00007f30cfe86738 00007f30cfe8b3d1 00007f30cfe8b389 00007f30cfe8b319 00007f30cfe8b2e1 00007f30cfe8a50d 00000000005bd8c4 00000000005bd6f9
0810 07:53:12.854942 (+ 27762us) op_driver.cc:464] REPLICATION: finished
0810 07:53:12.860959 (+ 6017us) write_op.cc:301] APPLY: starting
0810 07:53:12.861020 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:12.861111 (+ 91us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:12.861131 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:12.861304 (+ 173us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:12.861439 (+ 135us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:12.861483 (+ 44us) write_op.cc:312] APPLY: finished
0810 07:53:12.861703 (+ 220us) log.cc:844] Serialized 23 byte log entry
0810 07:53:12.861928 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:12.862021 (+ 93us) write_op.cc:454] Released schema lock
0810 07:53:12.862069 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:12.862422 (+ 353us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:12.862473 (+ 51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27748,"reported_tablets":34,"spinlock_wait_cycles":408663296,"child_traces":[["op",{"apply.queue_time_us":5825,"apply.run_cpu_time_us":1422,"apply.run_wall_time_us":1421,"num_ops":1,"prepare.queue_time_us":457838,"prepare.run_cpu_time_us":2764,"prepare.run_wall_time_us":33839,"raft.queue_time_us":1602,"raft.run_cpu_time_us":755,"raft.run_wall_time_us":756,"replication_time_us":60067,"spinlock_wait_cycles":23149440}]]}
I20260810 07:53:20.567832 3333 raft_consensus.cc:493] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.568378 3333 raft_consensus.cc:515] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.571484 3333 leader_election.cc:290] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:20.608971 380 tablet_service.cc:1558] Processing DeleteTablet for tablet c292cb975c504ad8abf03e913140c7a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 461d70a71f2c44228f4220e7f5fb9c64) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.610291 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 0098aaa7be644c9885167dc1ee4929b5 with delete_type TABLET_DATA_DELETED (replaced by tablet e1fa9200140a41dfbd886442ae336988) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.610899 3297 ts_tablet_manager.cc:1929] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:20.611457 3297 log.cc:1199] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/cec929b5f0e14f6586ed50137ecd741b
I20260810 07:53:20.612545 3271 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:20.618512 308 consensus_peers.cc:597] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:20.621816 308 leader_election.cc:343] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.622184 308 leader_election.cc:304] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:20.623328 3276 raft_consensus.cc:2749] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:20.625603 308 leader_election.cc:343] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.625967 308 leader_election.cc:304] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:20.626672 3188 leader_election.cc:290] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.627108 3276 raft_consensus.cc:2749] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:20.634268 1047 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:20.634713 1047 ts_tablet_manager.cc:1403] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.474s user 0.007s sys 0.007s
I20260810 07:53:20.629786 2740 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Refusing update from remote peer d86331b1fae14fb4b49b99f676a8490a: Log matching property violated. Preceding OpId in replica: term: 3 index: 394. Preceding OpId from leader: term: 5 index: 402. (index mismatch)
I20260810 07:53:20.649294 624 tablet_service.cc:1558] Processing DeleteTablet for tablet c9920a33cc93419ca3284a0b009236df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.650107 645 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.653268 3271 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.653383 1071 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:20.655548 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:57540
W20260810 07:53:20.556473 32468 rpcz_store.cc:269] 0810 07:53:12.221123 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.292471 (+ 71348us) service_pool.cc:224] Handling call
0810 07:53:12.829950 (+537479us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:20.529300 (+7699350us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b68000c5f60. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30d178ac01 00007f30d1767f5a 00007f30d1766698 00007f30d1764d50 00007f30d186b53e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c
0810 07:53:20.553132 (+ 23832us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:14.630988 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:14.631863 (+ 875us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:14.631901 (+ 38us) write_op.cc:435] Acquired schema lock
0810 07:53:14.631921 (+ 20us) tablet.cc:662] Decoding operations
0810 07:53:14.632496 (+ 575us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:14.756852 (+124356us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:14.756903 (+ 51us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:14.757184 (+ 281us) tablet.cc:701] Row locks acquired
0810 07:53:14.757203 (+ 19us) write_op.cc:260] PREPARE: finished
0810 07:53:14.757370 (+ 167us) write_op.cc:270] Start()
0810 07:53:14.757476 (+ 106us) write_op.cc:276] Timestamp: P: 1786348394757289 usec, L: 0
0810 07:53:14.757498 (+ 22us) op_driver.cc:348] REPLICATION: starting
0810 07:53:14.758222 (+ 724us) log.cc:844] Serialized 533 byte log entry
0810 07:53:15.322371 (+564149us) op_driver.cc:464] REPLICATION: finished
0810 07:53:15.339283 (+ 16912us) write_op.cc:301] APPLY: starting
0810 07:53:15.339358 (+ 75us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:15.339463 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:15.339583 (+ 120us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:15.339834 (+ 251us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:15.340014 (+ 180us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:15.340075 (+ 61us) write_op.cc:312] APPLY: finished
0810 07:53:15.340301 (+ 226us) log.cc:844] Serialized 23 byte log entry
0810 07:53:15.342988 (+ 2687us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:15.343068 (+ 80us) write_op.cc:454] Released schema lock
0810 07:53:15.343114 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:15.347759 (+ 4645us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:15.347809 (+ 50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":91342,"reported_tablets":35,"spinlock_wait_cycles":1169069184,"child_traces":[["op",{"apply.queue_time_us":16680,"apply.run_cpu_time_us":1768,"apply.run_wall_time_us":8395,"num_ops":1,"prepare.queue_time_us":160943,"prepare.run_cpu_time_us":7691,"prepare.run_wall_time_us":317984,"raft.queue_time_us":1160,"raft.run_cpu_time_us":896,"raft.run_wall_time_us":897,"replication_time_us":564744,"thread_start_us":332654,"threads_started":3,"wal-append.queue_time_us":194561,"wal-append.run_cpu_time_us":8909,"wal-append.run_wall_time_us":3481763}]]}
I20260810 07:53:20.656657 380 tablet_service.cc:1558] Processing DeleteTablet for tablet a04e29d3a49b4bd891be79e0fad6ec51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:10 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.657696 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 213fa4838fe847de87091cfba97eaedc with delete_type TABLET_DATA_DELETED (replaced by tablet f65480f06a3c4b2886aa2485f81e16f2) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.658100 3297 ts_tablet_manager.cc:1950] T cec929b5f0e14f6586ed50137ecd741b P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:20.658943 380 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.659219 3286 tablet_replica.cc:333] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:20.659931 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b48c88571040f6be8d7cbd388feb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:01 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.661119 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 7eeabaae541548afab4a3ff5ab388035 with delete_type TABLET_DATA_DELETED (replaced by tablet 1cb66183211f45d4a756f70c5e331f87) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.664392 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:20.665122 676 raft_consensus.cc:2393] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
W20260810 07:53:20.665429 574 leader_election.cc:343] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.666384 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:20.667096 411 raft_consensus.cc:2468] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:20.667243 3269 consensus_queue.cc:1048] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:20.667529 3064 consensus_queue.cc:1048] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:20.679471 1071 tablet_bootstrap.cc:654] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:20.683010 2560 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.31.139.125:46845 for RPC RequestConsensusVote timed out after 0.566s (ON_OUTBOUND_QUEUE)
W20260810 07:53:20.683605 32467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 46) took 9278 ms (client timeout 1000 ms). Trace:
W20260810 07:53:20.691713 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 35) took 8858 ms (client timeout 1000 ms). Trace:
I20260810 07:53:20.705235 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 8779af4159eb48da894e6a92c2c4a6f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 5a0f23d5212a46cc8f731d9c08019858) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:20.706501 622 tablet_service.cc:1558] Processing DeleteTablet for tablet a9be900b701a45728a44bf26350a26ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:08 UTC) from {username='slave'} at 127.0.0.1:57706
W20260810 07:53:20.590242 32461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.65:58899 (request call id 47) took 7850 ms (client timeout 1000 ms). Trace:
W20260810 07:53:20.615098 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260810 07:53:20.632151 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9b4339a26ad540f4b385f69fd0dcdc5d consensus state has no leader
I20260810 07:53:20.678380 2699 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" instance_seqno: 1786348360212850) as {username='slave'} at 127.31.139.65:47875; Asking this server to re-register.
W20260810 07:53:20.715019 2564 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.023s sys 0.004s
I20260810 07:53:20.724252 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 7000d25f3bfe4c99be54a0f4789cc96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:02 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.725409 380 tablet_service.cc:1558] Processing DeleteTablet for tablet ef32f7951f8640e9a13b069816b1cdd5 with delete_type TABLET_DATA_DELETED (replaced by tablet cca378598d0c45178e497fd212638d77) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.725919 3352 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:20.726339 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe822aa30b54fb091fd62ed8b6e22ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:58 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.726414 3352 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
W20260810 07:53:20.730171 32470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 36) took 7034 ms (client timeout 1000 ms). Trace:
I20260810 07:53:20.733306 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c5e309389d402092f59c801923854a with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb896c8922e412c97e2cc4891298830) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.734334 380 tablet_service.cc:1558] Processing DeleteTablet for tablet d301af14299040528cf3a033ff5177f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 4fb366297938415382cf6555ed052956) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.735270 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e0a84839427099a9da06e2b970ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:20.738052 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 394 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:20.739491 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet cec929b5f0e14f6586ed50137ecd741b (table default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]) successfully deleted
W20260810 07:53:20.741180 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 77216.3us
W20260810 07:53:20.730520 32470 rpcz_store.cc:269] 0810 07:53:13.695457 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:13.966898 (+271441us) service_pool.cc:224] Handling call
0810 07:53:20.730092 (+6763194us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:19.991157 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:19.991828 (+ 671us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:19.991866 (+ 38us) write_op.cc:435] Acquired schema lock
0810 07:53:19.991893 (+ 27us) tablet.cc:662] Decoding operations
0810 07:53:19.992498 (+ 605us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:19.992602 (+ 104us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:19.992641 (+ 39us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:19.992844 (+ 203us) tablet.cc:701] Row locks acquired
0810 07:53:19.992864 (+ 20us) write_op.cc:260] PREPARE: finished
0810 07:53:20.000641 (+ 7777us) write_op.cc:270] Start()
0810 07:53:20.000790 (+ 149us) write_op.cc:276] Timestamp: P: 1786348400000561 usec, L: 0
0810 07:53:20.000807 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:20.004688 (+ 3881us) log.cc:844] Serialized 533 byte log entry
0810 07:53:20.379353 (+374665us) op_driver.cc:464] REPLICATION: finished
0810 07:53:20.491028 (+111675us) write_op.cc:301] APPLY: starting
0810 07:53:20.491093 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:20.491196 (+ 103us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:20.491332 (+ 136us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:20.491538 (+ 206us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:20.491705 (+ 167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:20.491763 (+ 58us) write_op.cc:312] APPLY: finished
0810 07:53:20.491964 (+ 201us) log.cc:844] Serialized 23 byte log entry
0810 07:53:20.659896 (+167932us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:20.659977 (+ 81us) write_op.cc:454] Released schema lock
0810 07:53:20.660013 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:20.660299 (+ 286us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:20.660324 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":111469,"apply.run_cpu_time_us":1543,"apply.run_wall_time_us":169245,"num_ops":1,"prepare.queue_time_us":153564,"prepare.run_cpu_time_us":2979,"prepare.run_wall_time_us":20692,"replication_time_us":378417,"thread_start_us":113081,"threads_started":1}]]}
I20260810 07:53:20.742110 3333 consensus_queue.cc:1048] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:20.743664 792 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:20.744330 3333 raft_consensus.cc:2804] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:20.744663 3333 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:20.744992 3333 raft_consensus.cc:3060] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:20.752902 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:20.753475 411 raft_consensus.cc:2468] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:20.755093 3279 raft_consensus.cc:493] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.755544 3279 raft_consensus.cc:515] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.757529 3279 leader_election.cc:290] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.759652 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:20.760143 415 raft_consensus.cc:3060] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:20.720980 2690 ts_manager.cc:194] Registered new tserver with Master: 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:20.764252 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 81283.3us
W20260810 07:53:20.718146 3244 log.cc:927] Time spent T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260810 07:53:20.770383 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:20.772188 2564 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.654s user 0.023s sys 0.005s
I20260810 07:53:20.754701 32450 catalog_manager.cc:5050] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): tablet c9920a33cc93419ca3284a0b009236df (table default.Table_c849f5bbe83b49f1a5b41b6e28baacac [id=6d38995588ed44479ad04afe299ed43b]) successfully deleted
I20260810 07:53:20.769953 3286 raft_consensus.cc:2243] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:20.779498 3286 raft_consensus.cc:2272] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:20.780099 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:20.780365 3352 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:20.768666 500 heartbeater.cc:461] Registering TS with master...
I20260810 07:53:20.781617 3188 consensus_queue.cc:1048] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:20.776850 1071 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:20.785241 1071 ts_tablet_manager.cc:1403] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.132s user 0.007s sys 0.009s
I20260810 07:53:20.785760 3188 consensus_queue.cc:1048] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:20.786458 3286 ts_tablet_manager.cc:1916] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:20.772946 308 leader_election.cc:343] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.787834 1071 raft_consensus.cc:359] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.792397 1071 raft_consensus.cc:385] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:20.792670 1071 raft_consensus.cc:740] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.793243 1071 consensus_queue.cc:260] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.795249 1071 ts_tablet_manager.cc:1434] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260810 07:53:20.796156 1071 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:20.792521 2706 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" instance_seqno: 1786348362476985) as {username='slave'} at 127.31.139.66:55309; Asking this server to re-register.
I20260810 07:53:20.805409 415 raft_consensus.cc:2468] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:20.807297 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:20.810642 309 leader_election.cc:343] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.811475 309 leader_election.cc:304] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:20.813709 3280 raft_consensus.cc:2749] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:20.818991 3276 consensus_queue.cc:1048] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:20.821117 573 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:20.822691 3064 raft_consensus.cc:2804] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:20.823371 3064 raft_consensus.cc:697] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:20.830317 3318 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:53:20.832430 1071 tablet_bootstrap.cc:654] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:20.833097 3276 raft_consensus.cc:493] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.833902 3276 raft_consensus.cc:515] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.836977 3276 leader_election.cc:290] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:20.837613 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:20.824368 3064 consensus_queue.cc:237] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:20.839664 308 leader_election.cc:343] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.831274 2740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59478 (request call id 4) took 3846 ms. Trace:
I20260810 07:53:20.840966 2740 rpcz_store.cc:276] 0810 07:53:16.984591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:16.985360 (+ 769us) service_pool.cc:224] Handling call
0810 07:53:20.347633 (+3362273us) raft_consensus.cc:1471] Updating replica for 0 ops
0810 07:53:20.636460 (+288827us) raft_consensus.cc:1697] Filling consensus response to leader.
0810 07:53:20.830921 (+194461us) spinlock_profiling.cc:243] Waited 3.36 s on lock 0x7b54000104d0. stack: 00007f71bde8bcf3 00007f71bde8bbf0 00007f71bda39574 0000000000516de4 0000000000516d79 00007f71c4f8c951 00007f71c5022c8d 00007f71c6081229 00007f71c46ffd34 00007f71c46ffcc2 00007f71c46ffbf1 00007f71c46ffb6c 00007f71c46fee22 00007f71c2cee07c 00007f71c2ced4b6 00007f71c2cece5f
0810 07:53:20.831174 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7731703680}
I20260810 07:53:20.844071 3309 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 402, Last known committed idx: 392, Time since last communication: 0.000s
I20260810 07:53:20.844358 3318 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: loading cluster ID for follower catalog manager: success
I20260810 07:53:20.849014 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:20.851651 309 leader_election.cc:343] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.852181 309 leader_election.cc:304] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:20.856400 3283 ts_tablet_manager.cc:1929] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:20.857069 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:20.859149 32565 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:20.860008 3352 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:20.871657 3280 raft_consensus.cc:2749] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:20.873715 3297 tablet_replica.cc:333] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:20.876019 3283 log.cc:1199] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/cec929b5f0e14f6586ed50137ecd741b
I20260810 07:53:20.867933 742 heartbeater.cc:461] Registering TS with master...
I20260810 07:53:20.878247 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 4 candidate_status { last_received { term: 3 index: 394 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:20.866024 1047 raft_consensus.cc:359] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.879300 1047 raft_consensus.cc:385] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:20.879390 3297 raft_consensus.cc:2243] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:20.879632 1047 raft_consensus.cc:740] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.880290 1047 consensus_queue.cc:260] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.883677 1047 ts_tablet_manager.cc:1434] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.249s user 0.007s sys 0.000s
I20260810 07:53:20.887887 1047 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:20.887661 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:20.881705 3283 ts_tablet_manager.cc:1950] T cec929b5f0e14f6586ed50137ecd741b P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:20.895466 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 361) took 1241 ms. Trace:
I20260810 07:53:20.899592 1071 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:20.900233 1071 ts_tablet_manager.cc:1403] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.104s user 0.007s sys 0.008s
I20260810 07:53:20.903846 1071 raft_consensus.cc:359] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.905243 3283 rpcz_store.cc:276] 0810 07:53:19.653881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:19.654072 (+ 191us) service_pool.cc:224] Handling call
0810 07:53:20.316384 (+662312us) ts_tablet_manager.cc:1151] Deleting tablet cec929b5f0e14f6586ed50137ecd741b
0810 07:53:20.316428 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:20.749958 (+433530us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:20.856083 (+106125us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:20.895391 (+ 39308us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":661509}
I20260810 07:53:20.917994 3280 raft_consensus.cc:493] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:20.904567 1071 raft_consensus.cc:385] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:20.918877 1071 raft_consensus.cc:740] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:20.918540 3280 raft_consensus.cc:515] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.919622 1071 consensus_queue.cc:260] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:20.921952 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet cec929b5f0e14f6586ed50137ecd741b (table default.Table_64987f4480ff416b8bd9688ac5df15a5 [id=ef67ab598ae941228f12683878395e86]) successfully deleted
I20260810 07:53:20.922442 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:20.922712 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet a04e29d3a49b4bd891be79e0fad6ec51 because the tablet was not found. No further retry: Not found: Tablet not found: a04e29d3a49b4bd891be79e0fad6ec51
W20260810 07:53:20.923332 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7000d25f3bfe4c99be54a0f4789cc96c because the tablet was not found. No further retry: Not found: Tablet not found: 7000d25f3bfe4c99be54a0f4789cc96c
W20260810 07:53:20.925212 308 leader_election.cc:343] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:20.926801 1071 ts_tablet_manager.cc:1434] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.026s user 0.003s sys 0.003s
W20260810 07:53:20.932842 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 102201us
W20260810 07:53:20.933696 574 leader_election.cc:343] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:20.683943 32467 rpcz_store.cc:269] 0810 07:53:11.405290 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.424995 (+ 19705us) service_pool.cc:224] Handling call
0810 07:53:20.533168 (+9108173us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b68000c5f60. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30d178ac01 00007f30d1767f5a 00007f30d1766698 00007f30d1764d50 00007f30d186b53e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c
0810 07:53:20.683432 (+150264us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:12.971633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:12.972277 (+ 644us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:12.972303 (+ 26us) write_op.cc:435] Acquired schema lock
0810 07:53:12.972316 (+ 13us) tablet.cc:662] Decoding operations
0810 07:53:12.972667 (+ 351us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:12.972708 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:12.972774 (+ 66us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:12.973110 (+ 336us) tablet.cc:701] Row locks acquired
0810 07:53:12.973126 (+ 16us) write_op.cc:260] PREPARE: finished
0810 07:53:12.973264 (+ 138us) write_op.cc:270] Start()
0810 07:53:12.973357 (+ 93us) write_op.cc:276] Timestamp: P: 1786348392973117 usec, L: 0
0810 07:53:12.973374 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:12.973929 (+ 555us) log.cc:844] Serialized 913 byte log entry
0810 07:53:13.483820 (+509891us) op_driver.cc:464] REPLICATION: finished
0810 07:53:13.486789 (+ 2969us) write_op.cc:301] APPLY: starting
0810 07:53:13.486895 (+ 106us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:13.487059 (+ 164us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:13.487198 (+ 139us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:13.487632 (+ 434us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:13.487827 (+ 195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:13.487881 (+ 54us) write_op.cc:312] APPLY: finished
0810 07:53:13.488195 (+ 314us) log.cc:844] Serialized 29 byte log entry
0810 07:53:13.488434 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.488535 (+ 101us) write_op.cc:454] Released schema lock
0810 07:53:13.488574 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:13.515941 (+ 27367us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:13.515974 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":104251,"reported_tablets":38,"spinlock_wait_cycles":39260544,"child_traces":[["op",{"apply.queue_time_us":2649,"apply.run_cpu_time_us":2130,"apply.run_wall_time_us":29134,"num_ops":2,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":2564,"prepare.run_wall_time_us":2572,"replication_time_us":510178}]]}
W20260810 07:53:20.937639 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 50092.4us
I20260810 07:53:20.941988 3283 tablet_replica.cc:333] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:20.942999 1047 tablet_bootstrap.cc:654] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:20.966751 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:20.995958 3280 leader_election.cc:290] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:20.694952 32463 rpcz_store.cc:269] 0810 07:53:11.833115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:11.858621 (+ 25506us) service_pool.cc:224] Handling call
0810 07:53:20.691623 (+8833002us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:19.853172 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:19.853645 (+ 473us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:19.853673 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:19.853687 (+ 14us) tablet.cc:662] Decoding operations
0810 07:53:19.853968 (+ 281us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:19.854017 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:19.854056 (+ 39us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:19.854223 (+ 167us) tablet.cc:701] Row locks acquired
0810 07:53:19.854238 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:19.854379 (+ 141us) write_op.cc:270] Start()
0810 07:53:19.854463 (+ 84us) write_op.cc:276] Timestamp: P: 1786348399854315 usec, L: 0
0810 07:53:19.854480 (+ 17us) op_driver.cc:348] REPLICATION: starting
0810 07:53:19.855041 (+ 561us) log.cc:844] Serialized 356 byte log entry
0810 07:53:20.010282 (+155241us) op_driver.cc:464] REPLICATION: finished
0810 07:53:20.338807 (+328525us) write_op.cc:301] APPLY: starting
0810 07:53:20.338884 (+ 77us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:20.338998 (+ 114us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:20.339139 (+ 141us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:20.339390 (+ 251us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:20.339573 (+ 183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:20.339652 (+ 79us) write_op.cc:312] APPLY: finished
0810 07:53:20.339947 (+ 295us) log.cc:844] Serialized 23 byte log entry
0810 07:53:20.510598 (+170651us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:20.510665 (+ 67us) write_op.cc:454] Released schema lock
0810 07:53:20.510699 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:20.632590 (+121891us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:20.632620 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":74873,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":328315,"apply.run_cpu_time_us":1765,"apply.run_wall_time_us":293752,"num_ops":1,"prepare.queue_time_us":108603,"prepare.run_cpu_time_us":2266,"prepare.run_wall_time_us":137817,"raft.queue_time_us":103,"raft.run_cpu_time_us":723,"raft.run_wall_time_us":725,"replication_time_us":155674,"thread_start_us":476576,"threads_started":2}]]}
I20260810 07:53:21.006304 3280 consensus_queue.cc:1048] T c43aeafc19a240e6991b69249b0516fa P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:21.008903 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 213fa4838fe847de87091cfba97eaedc because the tablet was not found. No further retry: Not found: Tablet not found: 213fa4838fe847de87091cfba97eaedc
W20260810 07:53:21.011148 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 0098aaa7be644c9885167dc1ee4929b5 because the tablet was not found. No further retry: Not found: Tablet not found: 0098aaa7be644c9885167dc1ee4929b5
W20260810 07:53:21.012383 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 7eeabaae541548afab4a3ff5ab388035 because the tablet was not found. No further retry: Not found: Tablet not found: 7eeabaae541548afab4a3ff5ab388035
W20260810 07:53:21.013502 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet ef32f7951f8640e9a13b069816b1cdd5 because the tablet was not found. No further retry: Not found: Tablet not found: ef32f7951f8640e9a13b069816b1cdd5
W20260810 07:53:20.712311 32461 rpcz_store.cc:269] 0810 07:53:12.739876 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:12.940967 (+201091us) service_pool.cc:224] Handling call
0810 07:53:20.525773 (+7584806us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b68000c5f60. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30d178ac01 00007f30d1767f5a 00007f30d1766698 00007f30d1764d50 00007f30d186b53e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c
0810 07:53:20.590148 (+ 64375us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:15.404766 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:15.405349 (+ 583us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:15.405389 (+ 40us) write_op.cc:435] Acquired schema lock
0810 07:53:15.405412 (+ 23us) tablet.cc:662] Decoding operations
0810 07:53:15.405798 (+ 386us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:15.405852 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:15.405889 (+ 37us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:15.406099 (+ 210us) tablet.cc:701] Row locks acquired
0810 07:53:15.406123 (+ 24us) write_op.cc:260] PREPARE: finished
0810 07:53:15.406278 (+ 155us) write_op.cc:270] Start()
0810 07:53:15.406391 (+ 113us) write_op.cc:276] Timestamp: P: 1786348395406171 usec, L: 0
0810 07:53:15.406417 (+ 26us) op_driver.cc:348] REPLICATION: starting
0810 07:53:15.407138 (+ 721us) log.cc:844] Serialized 533 byte log entry
0810 07:53:17.188232 (+1781094us) op_driver.cc:464] REPLICATION: finished
0810 07:53:17.233145 (+ 44913us) write_op.cc:301] APPLY: starting
0810 07:53:17.233207 (+ 62us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:17.233354 (+ 147us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:17.233497 (+ 143us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:17.233983 (+ 486us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:17.234139 (+ 156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:17.234198 (+ 59us) write_op.cc:312] APPLY: finished
0810 07:53:17.304674 (+ 70476us) log.cc:844] Serialized 23 byte log entry
0810 07:53:17.305695 (+ 1021us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:17.305764 (+ 69us) write_op.cc:454] Released schema lock
0810 07:53:17.305798 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0810 07:53:17.326568 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:17.327204 (+ 636us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:17.327242 (+ 38us) write_op.cc:435] Acquired schema lock
0810 07:53:17.327265 (+ 23us) tablet.cc:662] Decoding operations
0810 07:53:17.327646 (+ 381us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:17.327707 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:17.327745 (+ 38us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:17.327956 (+ 211us) tablet.cc:701] Row locks acquired
0810 07:53:17.327974 (+ 18us) write_op.cc:260] PREPARE: finished
0810 07:53:17.328138 (+ 164us) write_op.cc:270] Start()
0810 07:53:17.328248 (+ 110us) write_op.cc:276] Timestamp: P: 1786348397328057 usec, L: 0
0810 07:53:17.328283 (+ 35us) op_driver.cc:348] REPLICATION: starting
0810 07:53:17.385083 (+ 56800us) log.cc:844] Serialized 356 byte log entry
0810 07:53:17.406185 (+ 21102us) op_driver.cc:464] REPLICATION: finished
0810 07:53:17.432935 (+ 26750us) write_op.cc:301] APPLY: starting
0810 07:53:17.433002 (+ 67us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:17.433099 (+ 97us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:17.433202 (+ 103us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:17.433404 (+ 202us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:17.433538 (+ 134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:17.433586 (+ 48us) write_op.cc:312] APPLY: finished
0810 07:53:17.433766 (+ 180us) log.cc:844] Serialized 23 byte log entry
0810 07:53:17.433936 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:17.434005 (+ 69us) write_op.cc:454] Released schema lock
0810 07:53:17.434037 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":59807,"reported_tablets":39,"spinlock_wait_cycles":23377664,"child_traces":[["op",{"apply.queue_time_us":44688,"apply.run_cpu_time_us":1756,"apply.run_wall_time_us":96635,"num_ops":1,"prepare.queue_time_us":136,"prepare.run_cpu_time_us":2682,"prepare.run_wall_time_us":2692,"replication_time_us":1781609}],["op",{"apply.queue_time_us":26563,"apply.run_cpu_time_us":1312,"apply.run_wall_time_us":1313,"num_ops":1,"prepare.queue_time_us":11543,"prepare.run_cpu_time_us":3081,"prepare.run_wall_time_us":60625,"raft.queue_time_us":9890,"raft.run_cpu_time_us":789,"raft.run_wall_time_us":790,"replication_time_us":77785}]]}
I20260810 07:53:21.030459 3039 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:53:21.054252 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.055435 3188 raft_consensus.cc:493] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.056020 3188 raft_consensus.cc:515] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.058243 3297 raft_consensus.cc:2272] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:21.058926 3188 leader_election.cc:290] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.061221 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.061911 410 raft_consensus.cc:2468] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.065230 573 leader_election.cc:304] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.066514 3188 raft_consensus.cc:2804] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.067101 3188 raft_consensus.cc:493] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.067724 3188 raft_consensus.cc:3060] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.074432 1047 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:21.076644 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.078787 1047 ts_tablet_manager.cc:1403] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.191s user 0.008s sys 0.015s
I20260810 07:53:21.079442 3297 ts_tablet_manager.cc:1916] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:21.084869 1064 tablet_bootstrap.cc:654] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.085032 1047 raft_consensus.cc:359] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.092912 1047 raft_consensus.cc:385] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.096374 1047 raft_consensus.cc:740] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.097637 1047 consensus_queue.cc:260] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.102810 1047 ts_tablet_manager.cc:1434] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260810 07:53:21.105969 1047 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:21.110512 309 leader_election.cc:343] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.111038 309 leader_election.cc:304] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:21.111902 3217 raft_consensus.cc:2749] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:21.113524 1071 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:20.897943 3333 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.123930 3333 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:21.128213 1071 tablet_bootstrap.cc:654] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.131261 1047 tablet_bootstrap.cc:654] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.138049 3388 tablet.cc:1724] T f6b54db926fa4032b6408d3d309de35b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:21.151499 3390 tablet.cc:1724] T f6b54db926fa4032b6408d3d309de35b P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:21.163022 1047 tablet_bootstrap.cc:492] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:21.163569 1047 ts_tablet_manager.cc:1403] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.058s user 0.010s sys 0.004s
I20260810 07:53:21.163357 3188 raft_consensus.cc:515] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.157155 1071 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:21.164494 1071 ts_tablet_manager.cc:1403] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.003s
I20260810 07:53:21.166153 1047 raft_consensus.cc:359] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.171020 1047 raft_consensus.cc:385] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.171367 1047 raft_consensus.cc:740] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.172159 1047 consensus_queue.cc:260] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.172981 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.173480 415 raft_consensus.cc:3060] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.173583 3188 leader_election.cc:290] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.174436 1047 ts_tablet_manager.cc:1434] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260810 07:53:21.175344 1047 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:21.159734 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 45839.3us
I20260810 07:53:21.180542 415 raft_consensus.cc:2468] T 9350b151227d45e6bae8adddd8fff86b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:21.190116 573 leader_election.cc:304] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.190881 3064 raft_consensus.cc:493] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.191298 3064 raft_consensus.cc:515] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.193298 3064 leader_election.cc:290] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.193804 3064 raft_consensus.cc:2804] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:21.195186 3064 raft_consensus.cc:697] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:21.195974 3064 consensus_queue.cc:237] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.196478 1071 raft_consensus.cc:359] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.197537 1071 raft_consensus.cc:385] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.197862 1071 raft_consensus.cc:740] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.198697 1071 consensus_queue.cc:260] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.125188 877 raft_consensus.cc:2393] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:21.203377 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.203881 1047 tablet_bootstrap.cc:654] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.204057 415 raft_consensus.cc:2468] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.147352 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.149482 3308 consensus_queue.cc:1048] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:21.150930 792 leader_election.cc:343] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:21.207755 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 56856.5us
I20260810 07:53:21.219688 573 leader_election.cc:304] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.220916 3188 raft_consensus.cc:2804] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.221280 3188 raft_consensus.cc:493] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.225423 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:21.225631 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:21.226333 3188 raft_consensus.cc:3060] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.235623 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.236404 415 raft_consensus.cc:2393] T 2c49060c69f2434db4e9f83f2147d7d1 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:21.246075 1071 ts_tablet_manager.cc:1434] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.079s user 0.006s sys 0.000s
I20260810 07:53:21.253597 1047 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:21.254130 1047 ts_tablet_manager.cc:1403] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.079s user 0.011s sys 0.003s
W20260810 07:53:21.254432 574 leader_election.cc:343] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.256814 792 leader_election.cc:304] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:21.258343 3308 raft_consensus.cc:2804] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.258831 3308 raft_consensus.cc:493] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.259364 3308 raft_consensus.cc:3060] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.251401 3188 raft_consensus.cc:515] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.257500 1047 raft_consensus.cc:359] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.270133 1047 raft_consensus.cc:385] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.270484 1047 raft_consensus.cc:740] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.273124 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.273320 1047 consensus_queue.cc:260] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.273684 415 raft_consensus.cc:3060] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.276624 3188 leader_election.cc:290] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.278090 1047 ts_tablet_manager.cc:1434] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260810 07:53:21.279304 1047 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:21.280404 3318 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: acquiring CA information for follower catalog manager: success
I20260810 07:53:21.291018 3308 raft_consensus.cc:515] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.292124 1064 tablet_bootstrap.cc:492] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:21.297955 3283 raft_consensus.cc:2243] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:21.298740 3283 raft_consensus.cc:2272] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:21.302269 3283 ts_tablet_manager.cc:1916] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:53:21.308254 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 50159.2us
I20260810 07:53:21.310216 1064 ts_tablet_manager.cc:1403] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.868s user 0.020s sys 0.001s
I20260810 07:53:21.318797 1047 tablet_bootstrap.cc:654] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.321725 1071 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:21.342257 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.348215 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.355485 1047 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:21.355973 1047 ts_tablet_manager.cc:1403] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.077s user 0.003s sys 0.012s
I20260810 07:53:21.361951 1064 raft_consensus.cc:359] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.362715 1064 raft_consensus.cc:385] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.362357 1047 raft_consensus.cc:359] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.363013 1064 raft_consensus.cc:740] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.363049 1047 raft_consensus.cc:385] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.363363 1047 raft_consensus.cc:740] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.363656 1064 consensus_queue.cc:260] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.365134 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.371717 1047 consensus_queue.cc:260] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.380762 3286 ts_tablet_manager.cc:1929] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:21.382755 1047 ts_tablet_manager.cc:1434] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.026s user 0.009s sys 0.000s
I20260810 07:53:21.388056 1047 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:21.396472 1064 ts_tablet_manager.cc:1434] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.065s user 0.006s sys 0.000s
I20260810 07:53:21.397693 1064 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:21.408385 3286 log.cc:1199] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/cec929b5f0e14f6586ed50137ecd741b
I20260810 07:53:21.409353 415 raft_consensus.cc:2468] T c6080c8b12014fe09d48e307d394f9c9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:53:21.415284 574 leader_election.cc:343] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:21.417559 574 leader_election.cc:343] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.409488 3286 ts_tablet_manager.cc:1950] T cec929b5f0e14f6586ed50137ecd741b P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:21.428973 573 leader_election.cc:304] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.430634 1047 tablet_bootstrap.cc:654] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.432962 3064 raft_consensus.cc:2804] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:21.433476 3064 raft_consensus.cc:697] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:21.434399 3064 consensus_queue.cc:237] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.449131 3373 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:53:21.454752 2701 ts_manager.cc:194] Registered new tserver with Master: 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:21.407913 1071 tablet_bootstrap.cc:654] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.458716 3414 tablet.cc:1724] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:21.465365 3286 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 321) took 1746 ms. Trace:
I20260810 07:53:21.465994 3286 rpcz_store.cc:276] 0810 07:53:19.718505 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:19.718721 (+ 216us) service_pool.cc:224] Handling call
0810 07:53:20.659039 (+940318us) ts_tablet_manager.cc:1151] Deleting tablet cec929b5f0e14f6586ed50137ecd741b
0810 07:53:20.659087 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.182418 (+523331us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.336569 (+154151us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.465239 (+128670us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109685,"tablet-delete.queue_time_us":931067}
W20260810 07:53:21.473069 792 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 51274.1us
I20260810 07:53:21.475051 3369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86331b1fae14fb4b49b99f676a8490a. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:21.476091 3369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:21.478367 2706 ts_manager.cc:194] Registered new tserver with Master: b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:21.493738 1047 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:21.494071 1064 tablet_bootstrap.cc:654] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.494248 1047 ts_tablet_manager.cc:1403] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.106s user 0.011s sys 0.006s
I20260810 07:53:21.495307 3308 leader_election.cc:290] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:21.505777 3188 raft_consensus.cc:493] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.506444 3188 raft_consensus.cc:515] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.509562 3188 leader_election.cc:290] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.512460 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.513087 415 raft_consensus.cc:2468] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.514461 573 leader_election.cc:304] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.515367 3188 raft_consensus.cc:2804] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.515769 3188 raft_consensus.cc:493] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.516119 3188 raft_consensus.cc:3060] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.523598 3113 raft_consensus.cc:493] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.524173 3113 raft_consensus.cc:515] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.525447 3297 ts_tablet_manager.cc:1929] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:21.525916 3297 log.cc:1199] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/c9920a33cc93419ca3284a0b009236df
I20260810 07:53:21.527621 3113 leader_election.cc:290] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.530030 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:21.531703 309 leader_election.cc:343] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.536810 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:21.539453 308 leader_election.cc:343] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.539871 308 leader_election.cc:304] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:21.544222 3279 raft_consensus.cc:2749] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:21.552809 308 consensus_peers.cc:597] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:21.557564 32455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 48970.9us
I20260810 07:53:21.553761 3271 raft_consensus.cc:493] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.561980 3271 raft_consensus.cc:515] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.567476 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.568167 415 raft_consensus.cc:2468] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.568914 3271 leader_election.cc:290] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.571358 573 leader_election.cc:304] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.571285 3188 raft_consensus.cc:515] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.573027 3271 raft_consensus.cc:2804] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.573541 3271 raft_consensus.cc:493] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.574281 3271 raft_consensus.cc:3060] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.586624 1064 tablet_bootstrap.cc:492] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:21.587175 1064 ts_tablet_manager.cc:1403] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.190s user 0.013s sys 0.000s
I20260810 07:53:21.590190 1047 raft_consensus.cc:359] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.590866 1047 raft_consensus.cc:385] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.591142 1047 raft_consensus.cc:740] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.596741 3297 ts_tablet_manager.cc:1950] T c9920a33cc93419ca3284a0b009236df P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
W20260810 07:53:21.598209 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 124391us
I20260810 07:53:21.608017 792 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:21.608830 3308 raft_consensus.cc:2749] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:21.609535 3279 raft_consensus.cc:493] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.610004 3279 raft_consensus.cc:515] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.613822 3349 raft_consensus.cc:493] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.614487 3349 raft_consensus.cc:515] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.619201 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.619827 412 raft_consensus.cc:2468] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.619828 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:21.621234 573 leader_election.cc:304] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.622014 3269 raft_consensus.cc:2804] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:21.622579 3269 raft_consensus.cc:493] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:21.623140 3269 raft_consensus.cc:3060] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.623610 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 313) took 1799 ms. Trace:
W20260810 07:53:21.629805 308 leader_election.cc:343] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.630393 3297 rpcz_store.cc:276] 0810 07:53:19.824447 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:19.825003 (+ 556us) service_pool.cc:224] Handling call
0810 07:53:20.873457 (+1048454us) ts_tablet_manager.cc:1151] Deleting tablet c9920a33cc93419ca3284a0b009236df
0810 07:53:20.873540 (+ 83us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.328084 (+454544us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.525309 (+197225us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.623517 (+ 98208us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1046919}
I20260810 07:53:21.632347 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 320) took 1094 ms. Trace:
I20260810 07:53:21.632789 3297 rpcz_store.cc:276] 0810 07:53:20.537571 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.537717 (+ 146us) service_pool.cc:224] Handling call
0810 07:53:21.632048 (+1094331us) ts_tablet_manager.cc:1151] Deleting tablet d301af14299040528cf3a033ff5177f8
0810 07:53:21.632102 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.632289 (+ 187us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1093731}
I20260810 07:53:21.634130 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 323) took 1091 ms. Trace:
I20260810 07:53:21.635517 3188 leader_election.cc:290] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.637117 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.637775 3279 leader_election.cc:290] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.637703 3297 rpcz_store.cc:276] 0810 07:53:20.542917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.543069 (+ 152us) service_pool.cc:224] Handling call
0810 07:53:21.633866 (+1090797us) ts_tablet_manager.cc:1151] Deleting tablet a04e29d3a49b4bd891be79e0fad6ec51
0810 07:53:21.633907 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.634080 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1090214}
I20260810 07:53:21.638167 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.639117 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 322) took 1096 ms. Trace:
I20260810 07:53:21.639693 412 raft_consensus.cc:3060] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.639463 3297 rpcz_store.cc:276] 0810 07:53:20.542628 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.543339 (+ 711us) service_pool.cc:224] Handling call
0810 07:53:21.638832 (+1095493us) ts_tablet_manager.cc:1151] Deleting tablet a9be900b701a45728a44bf26350a26ab
0810 07:53:21.638897 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.639053 (+ 156us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1094913}
W20260810 07:53:21.639957 309 leader_election.cc:343] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.640317 309 leader_election.cc:304] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:21.644822 3279 raft_consensus.cc:2749] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:21.645650 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.646337 415 raft_consensus.cc:3060] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.648924 412 raft_consensus.cc:2468] T 1fe71e69dfd74a3b826050bd7edc6b6a P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:21.626221 1064 raft_consensus.cc:359] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.650597 1064 raft_consensus.cc:385] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.650878 1064 raft_consensus.cc:740] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.651525 1064 consensus_queue.cc:260] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.654304 415 raft_consensus.cc:2468] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:21.663189 3199 raft_consensus.cc:493] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:21.663704 3199 raft_consensus.cc:515] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.666704 3199 leader_election.cc:290] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:21.668705 1064 ts_tablet_manager.cc:1434] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.081s user 0.005s sys 0.004s
I20260810 07:53:21.641296 3349 leader_election.cc:290] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.596087 3412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:21.676433 3412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:21.677166 3286 tablet_replica.cc:333] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
W20260810 07:53:21.644177 574 leader_election.cc:343] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:21.572247 3318 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:53:21.681532 1064 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:21.684595 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1fe71e69dfd74a3b826050bd7edc6b6a consensus state has no leader
I20260810 07:53:21.655095 1047 consensus_queue.cc:260] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.640636 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 324) took 1097 ms. Trace:
I20260810 07:53:21.689159 3297 rpcz_store.cc:276] 0810 07:53:20.543166 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.544040 (+ 874us) service_pool.cc:224] Handling call
0810 07:53:21.640382 (+1096342us) ts_tablet_manager.cc:1151] Deleting tablet 2d5ab03facf8477eada5d1ac3ec7f5ef
0810 07:53:21.640425 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.640589 (+ 164us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1095646}
I20260810 07:53:21.652349 573 leader_election.cc:304] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:21.698216 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 45982.4us
I20260810 07:53:21.701894 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 326) took 1147 ms. Trace:
I20260810 07:53:21.703263 3297 rpcz_store.cc:276] 0810 07:53:20.554865 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.554997 (+ 132us) service_pool.cc:224] Handling call
0810 07:53:21.701524 (+1146527us) ts_tablet_manager.cc:1151] Deleting tablet 8779af4159eb48da894e6a92c2c4a6f2
0810 07:53:21.701574 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.701820 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1145926}
I20260810 07:53:21.713361 3297 tablet_replica.cc:333] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:21.715737 3297 raft_consensus.cc:2243] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:21.714771 1064 tablet_bootstrap.cc:654] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.727994 3297 raft_consensus.cc:2272] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:21.732364 1071 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:21.732893 1071 ts_tablet_manager.cc:1403] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.411s user 0.011s sys 0.005s
I20260810 07:53:21.740198 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:21.735615 1071 raft_consensus.cc:359] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.745215 1071 raft_consensus.cc:385] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.745836 3269 raft_consensus.cc:515] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.747257 667 raft_consensus.cc:2393] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:21.718207 3271 raft_consensus.cc:515] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.752969 1064 tablet_bootstrap.cc:492] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:21.753666 3271 leader_election.cc:290] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.753927 792 leader_election.cc:304] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:21.754786 3405 raft_consensus.cc:2804] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:21.755324 3405 raft_consensus.cc:697] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:21.755556 1071 raft_consensus.cc:740] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.755179 3286 raft_consensus.cc:2243] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:21.756321 3405 consensus_queue.cc:237] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.757222 3286 raft_consensus.cc:2272] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:21.757462 3297 ts_tablet_manager.cc:1916] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:21.753468 1064 ts_tablet_manager.cc:1403] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.072s user 0.010s sys 0.000s
I20260810 07:53:21.763823 1064 raft_consensus.cc:359] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.764417 1064 raft_consensus.cc:385] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.767411 1071 consensus_queue.cc:260] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.770128 3269 leader_election.cc:290] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:21.744405 3188 raft_consensus.cc:2804] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:21.764665 1064 raft_consensus.cc:740] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.789028 1064 consensus_queue.cc:260] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:21.789860 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:21.791072 3283 ts_tablet_manager.cc:1929] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:21.795149 1071 ts_tablet_manager.cc:1434] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.062s user 0.005s sys 0.001s
I20260810 07:53:21.785028 3188 raft_consensus.cc:697] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:21.795578 1047 ts_tablet_manager.cc:1434] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.301s user 0.009s sys 0.002s
I20260810 07:53:21.796677 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.797381 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:21.797585 415 raft_consensus.cc:2393] T 4d6df1791e5146a38f3cf0304b5034cb P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:21.797906 412 raft_consensus.cc:3060] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.801071 1064 ts_tablet_manager.cc:1434] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260810 07:53:21.802034 1064 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:21.811998 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.812736 877 raft_consensus.cc:2468] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.812928 412 raft_consensus.cc:2468] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:21.815183 1047 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:21.815860 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.816573 877 raft_consensus.cc:2468] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:21.821528 3283 log.cc:1199] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:21.823342 3269 consensus_queue.cc:1048] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:21.824010 3283 ts_tablet_manager.cc:1950] T 8d92e0a84839427099a9da06e2b970ce P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:21.825654 573 leader_election.cc:304] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:21.837453 1064 tablet_bootstrap.cc:654] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:21.846889 3297 ts_tablet_manager.cc:1929] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:21.865468 1071 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:21.868856 3286 ts_tablet_manager.cc:1916] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:21.849653 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:21.853856 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.873302 886 raft_consensus.cc:3060] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.831887 1047 tablet_bootstrap.cc:654] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:21.878501 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 52926.1us
I20260810 07:53:21.835556 3188 consensus_queue.cc:237] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.880872 3271 raft_consensus.cc:2804] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:21.854983 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe71e69dfd74a3b826050bd7edc6b6a" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.859735 889 raft_consensus.cc:3060] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.881379 3271 raft_consensus.cc:697] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:21.882565 886 raft_consensus.cc:2468] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:21.882341 3271 consensus_queue.cc:237] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.860797 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6df1791e5146a38f3cf0304b5034cb" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:21.883558 3297 log.cc:1199] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/371fa2534984499b95c0945af358e8fc
I20260810 07:53:21.884531 3297 ts_tablet_manager.cc:1950] T 371fa2534984499b95c0945af358e8fc P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:21.847834 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 370) took 1579 ms. Trace:
I20260810 07:53:21.886466 3283 rpcz_store.cc:276] 0810 07:53:20.267780 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.267896 (+ 116us) service_pool.cc:224] Handling call
0810 07:53:20.941740 (+673844us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e0a84839427099a9da06e2b970ce
0810 07:53:20.941788 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.375970 (+434182us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.790908 (+414938us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.847372 (+ 56464us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341572,"tablet-delete.queue_time_us":673350}
I20260810 07:53:21.889882 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 371) took 1610 ms. Trace:
I20260810 07:53:21.892789 891 raft_consensus.cc:3060] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:21.887570 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 325) took 1333 ms. Trace:
I20260810 07:53:21.890368 3283 rpcz_store.cc:276] 0810 07:53:20.279094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.279218 (+ 124us) service_pool.cc:224] Handling call
0810 07:53:21.888337 (+1609119us) ts_tablet_manager.cc:1151] Deleting tablet 3fe822aa30b54fb091fd62ed8b6e22ff
0810 07:53:21.888403 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.888686 (+ 283us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1608468}
I20260810 07:53:21.898247 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 372) took 1618 ms. Trace:
I20260810 07:53:21.898703 3283 rpcz_store.cc:276] 0810 07:53:20.280142 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.280241 (+ 99us) service_pool.cc:224] Handling call
0810 07:53:21.897955 (+1617714us) ts_tablet_manager.cc:1151] Deleting tablet 45c5e309389d402092f59c801923854a
0810 07:53:21.898009 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.898185 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1617091}
I20260810 07:53:21.900606 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 377) took 1251 ms. Trace:
I20260810 07:53:21.903246 3283 rpcz_store.cc:276] 0810 07:53:20.648784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.649156 (+ 372us) service_pool.cc:224] Handling call
0810 07:53:21.900291 (+1251135us) ts_tablet_manager.cc:1151] Deleting tablet c9920a33cc93419ca3284a0b009236df
0810 07:53:21.900346 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.900542 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1250409}
I20260810 07:53:21.907845 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 376) took 1259 ms. Trace:
I20260810 07:53:21.899030 3297 rpcz_store.cc:276] 0810 07:53:20.554509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.555678 (+ 1169us) service_pool.cc:224] Handling call
0810 07:53:21.713172 (+1157494us) ts_tablet_manager.cc:1151] Deleting tablet 371fa2534984499b95c0945af358e8fc
0810 07:53:21.713217 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.805465 (+ 92248us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.846729 (+ 41264us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:21.887493 (+ 40764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21989,"tablet-delete.queue_time_us":1156972}
I20260810 07:53:21.937011 3297 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 327) took 1376 ms. Trace:
I20260810 07:53:21.939369 3297 rpcz_store.cc:276] 0810 07:53:20.560431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.560568 (+ 137us) service_pool.cc:224] Handling call
0810 07:53:21.936642 (+1376074us) ts_tablet_manager.cc:1151] Deleting tablet ef32f7951f8640e9a13b069816b1cdd5
0810 07:53:21.936697 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.936940 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1375434}
I20260810 07:53:21.941800 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:21.942606 411 raft_consensus.cc:2393] T 6cd9484084294acfba9613fa49f369b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:21.942970 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cd9484084294acfba9613fa49f369b2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:21.949354 1064 tablet_bootstrap.cc:492] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:21.949944 1064 ts_tablet_manager.cc:1403] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.148s user 0.012s sys 0.000s
I20260810 07:53:21.956219 891 raft_consensus.cc:2468] T 1fe71e69dfd74a3b826050bd7edc6b6a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:21.957109 1064 raft_consensus.cc:359] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.959194 1064 raft_consensus.cc:385] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:21.959566 1064 raft_consensus.cc:740] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:21.960516 1064 consensus_queue.cc:260] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:21.972990 3309 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "d86331b1fae14fb4b49b99f676a8490a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:21.973922 3309 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: LEADER
I20260810 07:53:21.912220 3283 rpcz_store.cc:276] 0810 07:53:20.648461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.649970 (+ 1509us) service_pool.cc:224] Handling call
0810 07:53:21.907469 (+1257499us) ts_tablet_manager.cc:1151] Deleting tablet d301af14299040528cf3a033ff5177f8
0810 07:53:21.907518 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.907763 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1256856}
I20260810 07:53:21.984292 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 378) took 1335 ms. Trace:
I20260810 07:53:21.984674 3283 rpcz_store.cc:276] 0810 07:53:20.649058 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.705080 (+ 56022us) service_pool.cc:224] Handling call
0810 07:53:21.983916 (+1278836us) ts_tablet_manager.cc:1151] Deleting tablet 8779af4159eb48da894e6a92c2c4a6f2
0810 07:53:21.983986 (+ 70us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.984232 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1278192}
I20260810 07:53:21.987001 3283 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 379) took 1280 ms. Trace:
I20260810 07:53:21.990267 1064 ts_tablet_manager.cc:1434] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260810 07:53:22.037943 1047 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:22.038425 1047 ts_tablet_manager.cc:1403] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.223s user 0.011s sys 0.013s
I20260810 07:53:21.991397 1064 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:21.987803 3283 rpcz_store.cc:276] 0810 07:53:20.706215 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.706324 (+ 109us) service_pool.cc:224] Handling call
0810 07:53:21.986524 (+1280200us) ts_tablet_manager.cc:1151] Deleting tablet a9be900b701a45728a44bf26350a26ab
0810 07:53:21.986594 (+ 70us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:21.986926 (+ 332us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1279460}
I20260810 07:53:22.009341 792 leader_election.cc:304] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:22.064693 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 55436.6us
I20260810 07:53:22.067253 3405 raft_consensus.cc:2749] T 6cd9484084294acfba9613fa49f369b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:22.066720 1047 raft_consensus.cc:359] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.068406 1047 raft_consensus.cc:385] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.068742 1047 raft_consensus.cc:740] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.069478 1047 consensus_queue.cc:260] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.085098 1071 tablet_bootstrap.cc:654] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:22.095391 3447 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:22.096153 3447 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:22.112290 1047 ts_tablet_manager.cc:1434] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.074s user 0.007s sys 0.001s
W20260810 07:53:22.124548 3435 log.cc:927] Time spent T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.197s user 0.001s sys 0.000s
I20260810 07:53:22.131825 3447 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:22.144891 3412 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:22.145478 3412 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:22.147218 1064 tablet_bootstrap.cc:654] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:22.158802 1047 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:22.174767 1047 tablet_bootstrap.cc:654] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:22.180186 3412 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:22.189445 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:22.199548 1071 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:22.209553 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:22.228621 1071 ts_tablet_manager.cc:1403] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.363s user 0.000s sys 0.014s
I20260810 07:53:22.241206 3286 ts_tablet_manager.cc:1929] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:22.246104 3286 log.cc:1199] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:22.247547 3286 ts_tablet_manager.cc:1950] T 8d92e0a84839427099a9da06e2b970ce P 60781ef3c8b84a8788f43d77e1bb3b19: Deleting consensus metadata
I20260810 07:53:22.256808 3286 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57540 (request call id 344) took 1521 ms. Trace:
I20260810 07:53:22.258993 3405 raft_consensus.cc:493] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.259837 3405 raft_consensus.cc:515] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.259859 3286 rpcz_store.cc:276] 0810 07:53:20.735046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:20.735144 (+ 98us) service_pool.cc:224] Handling call
0810 07:53:21.675364 (+940220us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e0a84839427099a9da06e2b970ce
0810 07:53:21.675422 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:22.155738 (+480316us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:22.241043 (+ 85305us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:22.256707 (+ 15664us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76560,"tablet-delete.queue_time_us":939621}
I20260810 07:53:22.273699 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:22.276984 1047 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:22.276877 3405 leader_election.cc:290] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:22.277572 1047 ts_tablet_manager.cc:1403] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.119s user 0.017s sys 0.016s
I20260810 07:53:22.280311 1064 tablet_bootstrap.cc:492] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:22.280885 1064 ts_tablet_manager.cc:1403] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.290s user 0.015s sys 0.000s
I20260810 07:53:22.281664 1047 raft_consensus.cc:359] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.282356 1047 raft_consensus.cc:385] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.282742 1047 raft_consensus.cc:740] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.284194 1047 consensus_queue.cc:260] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:22.294817 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:22.294893 1047 ts_tablet_manager.cc:1434] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260810 07:53:22.295898 1047 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:22.232438 1071 raft_consensus.cc:359] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.301251 1071 raft_consensus.cc:385] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.301635 1071 raft_consensus.cc:740] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.302464 1071 consensus_queue.cc:260] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.319171 1064 raft_consensus.cc:359] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.320441 1071 ts_tablet_manager.cc:1434] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.091s user 0.007s sys 0.000s
I20260810 07:53:22.320830 1064 raft_consensus.cc:385] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.321187 1064 raft_consensus.cc:740] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.322280 1071 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:22.324093 1064 consensus_queue.cc:260] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.354184 32301 master-stress-test.cc:579] Killing test thread 1/10
I20260810 07:53:22.417541 1064 ts_tablet_manager.cc:1434] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.136s user 0.006s sys 0.000s
I20260810 07:53:22.418919 1064 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:22.464816 1071 tablet_bootstrap.cc:654] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:22.179700 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 218591us
W20260810 07:53:22.176661 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 210693us
I20260810 07:53:22.490958 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d717f39a6704199abc34b220d795e48" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:22.500308 1047 tablet_bootstrap.cc:700] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: Time spent opening tablet: real 0.199s user 0.001s sys 0.000s
I20260810 07:53:22.502023 1047 tablet_bootstrap.cc:654] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:22.507642 792 leader_election.cc:304] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:22.508906 3405 raft_consensus.cc:2749] T 8d717f39a6704199abc34b220d795e48 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:22.528112 3113 raft_consensus.cc:493] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.528645 3113 raft_consensus.cc:515] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.545333 3422 raft_consensus.cc:493] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.546198 3422 raft_consensus.cc:515] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.549039 3422 leader_election.cc:290] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:22.553228 574 leader_election.cc:304] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:22.555727 3271 raft_consensus.cc:2749] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:22.556357 3269 consensus_queue.cc:1048] T 8d717f39a6704199abc34b220d795e48 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:22.572150 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:22.573910 309 leader_election.cc:343] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:22.574828 3113 leader_election.cc:290] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:22.577426 1064 tablet_bootstrap.cc:654] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:22.580552 3449 consensus_queue.cc:1048] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:22.603084 1071 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:22.603632 1071 ts_tablet_manager.cc:1403] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.282s user 0.011s sys 0.000s
I20260810 07:53:22.614944 3113 raft_consensus.cc:493] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.615549 3113 raft_consensus.cc:515] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.614362 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:22.620184 3449 consensus_queue.cc:1048] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:22.620530 308 leader_election.cc:343] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:22.620967 308 leader_election.cc:304] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:22.621631 3217 raft_consensus.cc:2749] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:22.628137 309 consensus_peers.cc:597] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 371fa2534984499b95c0945af358e8fc. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:22.629798 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:22.628247 3113 leader_election.cc:290] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:22.631573 309 leader_election.cc:343] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:22.655602 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:22.657321 410 raft_consensus.cc:2468] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:22.661334 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:22.675221 32563 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.544s (SENT) [suppressed 1 similar messages]
W20260810 07:53:22.694780 308 leader_election.cc:343] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:22.695528 308 leader_election.cc:304] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:22.696755 3217 raft_consensus.cc:2749] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:22.715520 1071 raft_consensus.cc:359] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.716235 1071 raft_consensus.cc:385] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.667986 3271 consensus_queue.cc:1048] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20260810 07:53:22.727725 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:22.742585 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 47908.8us
I20260810 07:53:22.691666 3422 consensus_queue.cc:1048] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.067s
I20260810 07:53:22.769675 573 leader_election.cc:304] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:22.770219 3367 consensus_queue.cc:1048] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:22.772053 3367 raft_consensus.cc:2804] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:22.772444 3367 raft_consensus.cc:493] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:22.774590 3367 raft_consensus.cc:3060] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:22.752950 1064 tablet_bootstrap.cc:492] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:22.785205 1064 ts_tablet_manager.cc:1403] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.367s user 0.008s sys 0.008s
I20260810 07:53:22.788017 1064 raft_consensus.cc:359] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.790947 3449 raft_consensus.cc:493] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.716519 1071 raft_consensus.cc:740] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.791473 3449 raft_consensus.cc:515] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.793710 3449 leader_election.cc:290] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:22.796185 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:22.802332 1047 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
W20260810 07:53:22.809844 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 51985.1us
I20260810 07:53:22.810037 3349 consensus_queue.cc:1048] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:22.812156 1047 ts_tablet_manager.cc:1403] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.516s user 0.019s sys 0.003s
I20260810 07:53:22.830961 1064 raft_consensus.cc:385] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.831373 1064 raft_consensus.cc:740] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.832082 1064 consensus_queue.cc:260] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.838624 3349 consensus_queue.cc:1048] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:22.848505 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:22.850481 792 leader_election.cc:304] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:22.853631 3449 raft_consensus.cc:2749] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:22.857337 1064 ts_tablet_manager.cc:1434] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.072s user 0.006s sys 0.001s
W20260810 07:53:22.861907 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:22.865468 3367 raft_consensus.cc:515] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.875988 1047 raft_consensus.cc:359] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.924007 1071 consensus_queue.cc:260] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.938184 3425 raft_consensus.cc:493] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:22.900522 1047 raft_consensus.cc:385] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:22.940150 1047 raft_consensus.cc:740] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:22.938849 3425 raft_consensus.cc:515] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:22.952790 3425 leader_election.cc:290] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:22.941140 1047 consensus_queue.cc:260] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.014308 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:23.015012 411 raft_consensus.cc:2468] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:23.034536 3271 raft_consensus.cc:493] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:23.035730 3271 raft_consensus.cc:515] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.038621 3271 leader_election.cc:290] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:23.047122 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 99195.8us
I20260810 07:53:23.057919 1064 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:23.063968 1064 tablet_bootstrap.cc:654] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.066250 1071 ts_tablet_manager.cc:1434] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.456s user 0.008s sys 0.000s
I20260810 07:53:23.083568 891 raft_consensus.cc:3060] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.080526 1047 ts_tablet_manager.cc:1434] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.262s user 0.020s sys 0.002s
W20260810 07:53:23.121791 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46595.1us
I20260810 07:53:23.141189 1064 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:23.141661 1064 ts_tablet_manager.cc:1403] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.007s
I20260810 07:53:23.085989 3367 leader_election.cc:290] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:23.144043 1064 raft_consensus.cc:359] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.144681 1064 raft_consensus.cc:385] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.146180 1064 raft_consensus.cc:740] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.146916 1064 consensus_queue.cc:260] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.148379 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:23.149219 889 raft_consensus.cc:2468] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:23.170846 1071 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:23.177610 1071 tablet_bootstrap.cc:654] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.183140 1064 ts_tablet_manager.cc:1434] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.039s user 0.008s sys 0.003s
I20260810 07:53:23.197396 1047 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:23.202118 3480 consensus_queue.cc:1048] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:23.213297 574 consensus_peers.cc:597] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:23.215627 574 leader_election.cc:343] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:23.236035 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.544s (SENT)
I20260810 07:53:23.236476 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:23.258134 3497 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:23.262571 3442 consensus_queue.cc:1048] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:23.264689 1064 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:23.292217 3271 consensus_queue.cc:1048] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:23.298905 1071 tablet_bootstrap.cc:492] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:23.299489 1071 ts_tablet_manager.cc:1403] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.129s user 0.009s sys 0.005s
I20260810 07:53:23.340972 3271 consensus_queue.cc:1048] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:23.342602 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 46077.5us
I20260810 07:53:23.343431 574 leader_election.cc:304] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:23.344202 3271 raft_consensus.cc:2804] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:23.344542 3271 raft_consensus.cc:493] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:23.344980 3271 raft_consensus.cc:3060] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:23.348680 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:23.302824 1071 raft_consensus.cc:359] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.361258 1071 raft_consensus.cc:385] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.361608 1071 raft_consensus.cc:740] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.362581 1071 consensus_queue.cc:260] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:23.365592 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.548s (SENT)
I20260810 07:53:23.366060 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:23.358184 3271 raft_consensus.cc:515] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.374495 3271 leader_election.cc:290] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:23.333832 1047 tablet_bootstrap.cc:654] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.381371 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:23.383929 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:23.384469 885 raft_consensus.cc:3060] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.387952 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:23.388558 411 raft_consensus.cc:2468] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:23.410995 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:23.411928 877 raft_consensus.cc:3060] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.423571 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9959255b4d46e29b9164027638e38e" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:23.424220 411 raft_consensus.cc:3060] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.429584 3405 raft_consensus.cc:493] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:23.430368 3405 raft_consensus.cc:515] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.431908 411 raft_consensus.cc:2468] T cc9959255b4d46e29b9164027638e38e P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:23.433589 3405 leader_election.cc:290] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:23.440404 3498 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:23.441555 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 76041.5us
W20260810 07:53:23.444644 574 leader_election.cc:343] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:23.449467 1071 ts_tablet_manager.cc:1434] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.150s user 0.005s sys 0.001s
I20260810 07:53:23.451932 1071 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:23.454421 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:23.457108 677 raft_consensus.cc:2468] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:23.471477 1064 tablet_bootstrap.cc:654] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.448495 885 raft_consensus.cc:2468] T cc9959255b4d46e29b9164027638e38e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:23.506013 1064 tablet_bootstrap.cc:492] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:23.507599 1064 ts_tablet_manager.cc:1403] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.243s user 0.017s sys 0.008s
I20260810 07:53:23.514086 1064 raft_consensus.cc:359] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.514918 1064 raft_consensus.cc:385] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.515204 1064 raft_consensus.cc:740] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.515955 1064 consensus_queue.cc:260] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.518505 1064 ts_tablet_manager.cc:1434] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
W20260810 07:53:23.532977 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 64147.3us
I20260810 07:53:23.544375 792 leader_election.cc:304] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:23.464850 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:23.545581 3484 raft_consensus.cc:2804] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:23.467515 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:23.545967 3484 raft_consensus.cc:493] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:23.546114 411 raft_consensus.cc:2468] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:23.546146 409 raft_consensus.cc:3060] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.546358 3484 raft_consensus.cc:3060] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.554538 409 raft_consensus.cc:2468] T a4863f3f03104d31ad4dafda6f33c9af P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:23.573314 877 raft_consensus.cc:2468] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:23.571720 3484 raft_consensus.cc:515] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.596060 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:23.596916 409 raft_consensus.cc:3060] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.615875 1071 tablet_bootstrap.cc:654] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.619004 1064 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:23.626135 1064 tablet_bootstrap.cc:654] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.630904 3484 leader_election.cc:290] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:23.664404 1064 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:23.664937 1064 ts_tablet_manager.cc:1403] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.046s user 0.013s sys 0.001s
I20260810 07:53:23.667549 1064 raft_consensus.cc:359] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.668258 1064 raft_consensus.cc:385] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.668540 1064 raft_consensus.cc:740] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.681037 1064 consensus_queue.cc:260] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.683271 3484 raft_consensus.cc:493] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:23.684127 3484 raft_consensus.cc:515] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.686506 1064 ts_tablet_manager.cc:1434] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260810 07:53:23.682487 409 raft_consensus.cc:2468] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:23.688366 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:23.736292 3484 leader_election.cc:290] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:23.738181 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 292705us
I20260810 07:53:23.741940 792 leader_election.cc:304] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:23.743741 1047 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:23.744189 1047 ts_tablet_manager.cc:1403] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.547s user 0.023s sys 0.033s
I20260810 07:53:23.746743 1047 raft_consensus.cc:359] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.747486 1047 raft_consensus.cc:385] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.747767 1047 raft_consensus.cc:740] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.748379 1047 consensus_queue.cc:260] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.750000 1047 ts_tablet_manager.cc:1434] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:53:23.750982 1047 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:23.756421 574 leader_election.cc:304] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:23.757718 3514 raft_consensus.cc:2804] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:23.758149 3484 raft_consensus.cc:2804] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:23.758219 3514 raft_consensus.cc:697] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:23.758993 3484 raft_consensus.cc:697] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:23.759104 3514 consensus_queue.cc:237] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.759864 3484 consensus_queue.cc:237] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.774597 1064 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:23.760890 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 167019us
I20260810 07:53:23.779870 573 leader_election.cc:304] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:23.781838 3516 raft_consensus.cc:2804] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:23.782207 3516 raft_consensus.cc:493] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:23.782509 3516 raft_consensus.cc:3060] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.784235 574 leader_election.cc:304] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:23.785036 3349 raft_consensus.cc:2804] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:23.785431 1064 tablet_bootstrap.cc:654] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.785517 3349 raft_consensus.cc:697] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:23.786386 3349 consensus_queue.cc:237] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.799351 1071 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:23.819232 1071 ts_tablet_manager.cc:1403] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.368s user 0.012s sys 0.006s
I20260810 07:53:23.839474 1064 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:23.840163 1064 ts_tablet_manager.cc:1403] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.003s
I20260810 07:53:23.843106 1064 raft_consensus.cc:359] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.843880 1064 raft_consensus.cc:385] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.844202 1064 raft_consensus.cc:740] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:23.847149 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:23.865618 1071 raft_consensus.cc:359] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.866550 1071 raft_consensus.cc:385] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:23.866912 1071 raft_consensus.cc:740] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:23.867621 1071 consensus_queue.cc:260] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.869995 1071 ts_tablet_manager.cc:1434] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.044s user 0.003s sys 0.002s
I20260810 07:53:23.871466 1047 tablet_bootstrap.cc:654] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:23.876533 889 raft_consensus.cc:3060] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.875412 3516 raft_consensus.cc:515] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:23.892370 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 110720us
I20260810 07:53:23.905921 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:23.906498 666 raft_consensus.cc:3060] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:23.906961 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9350b151227d45e6bae8adddd8fff86b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:23.916754 666 raft_consensus.cc:2468] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:23.918578 792 leader_election.cc:304] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:23.923532 32455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 50795.7us
I20260810 07:53:23.886129 1064 consensus_queue.cc:260] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:23.930903 1064 ts_tablet_manager.cc:1434] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.090s user 0.000s sys 0.006s
I20260810 07:53:23.931965 3516 leader_election.cc:290] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:23.946830 1047 tablet_bootstrap.cc:492] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:23.947335 1047 ts_tablet_manager.cc:1403] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.197s user 0.013s sys 0.000s
I20260810 07:53:23.956745 1064 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:23.958669 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:23.960486 3405 raft_consensus.cc:2749] T 9350b151227d45e6bae8adddd8fff86b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:24.001708 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:24.002337 410 raft_consensus.cc:3060] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.012461 410 raft_consensus.cc:2468] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:53:24.019523 574 leader_election.cc:343] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.048454 1047 raft_consensus.cc:359] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.049466 1047 raft_consensus.cc:385] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.049824 1047 raft_consensus.cc:740] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.052750 1071 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:24.053960 1047 consensus_queue.cc:260] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.056007 1047 ts_tablet_manager.cc:1434] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.108s user 0.006s sys 0.000s
I20260810 07:53:24.061911 1071 tablet_bootstrap.cc:654] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.076954 573 leader_election.cc:304] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:24.077728 3367 raft_consensus.cc:2804] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.078222 3367 raft_consensus.cc:697] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:24.079058 3367 consensus_queue.cc:237] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.114650 1064 tablet_bootstrap.cc:654] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:24.157286 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 51879.8us
I20260810 07:53:24.180284 1071 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:24.180841 1071 ts_tablet_manager.cc:1403] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.128s user 0.004s sys 0.018s
I20260810 07:53:24.160508 1047 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:24.183486 1071 raft_consensus.cc:359] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.184228 1071 raft_consensus.cc:385] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.184561 1071 raft_consensus.cc:740] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.185276 1071 consensus_queue.cc:260] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.188943 1071 ts_tablet_manager.cc:1434] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260810 07:53:24.191561 1071 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:24.212196 1071 tablet_bootstrap.cc:654] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.217118 1047 tablet_bootstrap.cc:654] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:24.251935 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:24.269598 1064 tablet_bootstrap.cc:492] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:24.270195 1064 ts_tablet_manager.cc:1403] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.314s user 0.021s sys 0.001s
I20260810 07:53:24.272046 1047 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:24.272557 1047 ts_tablet_manager.cc:1403] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.167s user 0.011s sys 0.000s
I20260810 07:53:24.275532 1047 raft_consensus.cc:359] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.279619 1071 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:24.280315 1071 ts_tablet_manager.cc:1403] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.089s user 0.011s sys 0.000s
I20260810 07:53:24.283412 1071 raft_consensus.cc:359] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.294121 1071 raft_consensus.cc:385] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.294497 1071 raft_consensus.cc:740] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.295238 1071 consensus_queue.cc:260] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.297097 1064 raft_consensus.cc:359] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.297616 1071 ts_tablet_manager.cc:1434] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260810 07:53:24.297845 1064 raft_consensus.cc:385] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.298123 1064 raft_consensus.cc:740] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.298840 1064 consensus_queue.cc:260] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.301005 1064 ts_tablet_manager.cc:1434] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260810 07:53:24.301760 1047 raft_consensus.cc:385] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.305877 3405 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:24.306524 1047 raft_consensus.cc:740] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.306403 3405 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:24.308287 3405 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:24.309968 1071 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:24.311810 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
W20260810 07:53:24.312906 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:24.313283 792 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:24.314635 3405 raft_consensus.cc:2749] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:24.317612 1064 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:24.307301 1047 consensus_queue.cc:260] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.357193 3442 raft_consensus.cc:493] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.357718 3442 raft_consensus.cc:515] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.360008 3442 leader_election.cc:290] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.362071 1047 ts_tablet_manager.cc:1434] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.089s user 0.006s sys 0.000s
I20260810 07:53:24.364751 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:24.365432 889 raft_consensus.cc:2468] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:24.365911 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:24.366536 411 raft_consensus.cc:2468] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:24.373467 574 leader_election.cc:304] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:24.376338 1047 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:24.377058 3515 raft_consensus.cc:2804] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.377429 3515 raft_consensus.cc:493] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.377765 3515 raft_consensus.cc:3060] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.381114 1071 tablet_bootstrap.cc:654] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.398056 1047 tablet_bootstrap.cc:654] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.402685 1071 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:24.403131 1071 ts_tablet_manager.cc:1403] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.001s
I20260810 07:53:24.406329 1071 raft_consensus.cc:359] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.407021 1071 raft_consensus.cc:385] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.409104 1071 raft_consensus.cc:740] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.417027 1071 consensus_queue.cc:260] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.418939 1071 ts_tablet_manager.cc:1434] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260810 07:53:24.419929 1071 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:24.473114 1064 tablet_bootstrap.cc:654] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.495196 3515 raft_consensus.cc:515] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.499754 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:24.500447 410 raft_consensus.cc:3060] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.504484 1071 tablet_bootstrap.cc:654] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.509707 1047 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:24.510123 1047 ts_tablet_manager.cc:1403] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.134s user 0.017s sys 0.000s
I20260810 07:53:24.512802 1047 raft_consensus.cc:359] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.513492 1047 raft_consensus.cc:385] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.513764 1047 raft_consensus.cc:740] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.532070 410 raft_consensus.cc:2468] T 77fa93c49cb54e5fa03034eda6fc2300 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:24.533552 573 leader_election.cc:304] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:24.534370 3442 raft_consensus.cc:2804] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.534850 3442 raft_consensus.cc:697] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:24.535415 1047 consensus_queue.cc:260] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.535601 3442 consensus_queue.cc:237] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.536440 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fa93c49cb54e5fa03034eda6fc2300" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:24.537295 889 raft_consensus.cc:3060] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.542708 3367 raft_consensus.cc:493] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.557523 3367 raft_consensus.cc:515] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.560523 3367 leader_election.cc:290] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.560750 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849197942f134542a387a10167fc7481" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:24.561681 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849197942f134542a387a10167fc7481" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:24.562094 574 leader_election.cc:343] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.562842 410 raft_consensus.cc:2468] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:24.563030 1047 ts_tablet_manager.cc:1434] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260810 07:53:24.563685 1071 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:24.564174 1071 ts_tablet_manager.cc:1403] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.144s user 0.014s sys 0.000s
I20260810 07:53:24.564172 573 leader_election.cc:304] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:24.536638 3515 leader_election.cc:290] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.566504 3515 raft_consensus.cc:2804] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.566892 3515 raft_consensus.cc:493] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.567273 3515 raft_consensus.cc:3060] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.570140 1047 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:24.572855 1071 raft_consensus.cc:359] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.573510 1071 raft_consensus.cc:385] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.573768 1071 raft_consensus.cc:740] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.577558 1071 consensus_queue.cc:260] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.583436 3515 raft_consensus.cc:515] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.586803 889 raft_consensus.cc:2468] T 77fa93c49cb54e5fa03034eda6fc2300 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:24.591068 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849197942f134542a387a10167fc7481" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:24.591660 410 raft_consensus.cc:3060] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.600278 410 raft_consensus.cc:2468] T 849197942f134542a387a10167fc7481 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:24.590531 1064 tablet_bootstrap.cc:492] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:24.601822 573 leader_election.cc:304] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:24.602284 1064 ts_tablet_manager.cc:1403] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.285s user 0.007s sys 0.007s
I20260810 07:53:24.602659 3367 raft_consensus.cc:2804] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.603170 3367 raft_consensus.cc:697] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:24.603997 3367 consensus_queue.cc:237] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.609798 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849197942f134542a387a10167fc7481" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:24.608697 1047 tablet_bootstrap.cc:654] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:24.611110 574 leader_election.cc:343] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.609584 3515 leader_election.cc:290] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.647038 3555 raft_consensus.cc:493] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.648027 3555 raft_consensus.cc:515] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.652508 3217 raft_consensus.cc:493] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.652946 3217 raft_consensus.cc:515] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.654368 3555 leader_election.cc:290] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:24.655023 3217 leader_election.cc:290] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.655326 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:24.659673 792 leader_election.cc:343] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.659797 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:24.660343 409 raft_consensus.cc:2468] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:24.660566 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:24.660903 1064 raft_consensus.cc:359] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.661146 666 raft_consensus.cc:2468] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:24.661545 1064 raft_consensus.cc:385] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.662087 308 leader_election.cc:304] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:24.662295 1064 raft_consensus.cc:740] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.663327 3113 raft_consensus.cc:2804] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.662344 792 leader_election.cc:304] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:24.663619 3113 raft_consensus.cc:493] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.663921 3113 raft_consensus.cc:3060] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.663962 3555 raft_consensus.cc:2804] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.664310 3555 raft_consensus.cc:493] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.664662 3555 raft_consensus.cc:3060] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:24.665151 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:24.671391 3217 raft_consensus.cc:493] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.671315 3113 raft_consensus.cc:515] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.677564 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:24.671797 3217 raft_consensus.cc:515] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:24.680461 309 leader_election.cc:343] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.682962 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:24.683477 666 raft_consensus.cc:3060] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.688970 3558 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:24.693056 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:24.693789 3113 leader_election.cc:290] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.694496 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:24.695108 677 raft_consensus.cc:2468] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:24.695863 309 leader_election.cc:343] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.698228 3558 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:24.702793 308 leader_election.cc:304] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:24.703634 3113 raft_consensus.cc:2804] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.704854 3113 raft_consensus.cc:493] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.705246 3113 raft_consensus.cc:3060] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.695479 1071 ts_tablet_manager.cc:1434] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.131s user 0.003s sys 0.003s
I20260810 07:53:24.711148 1071 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:24.716463 3555 raft_consensus.cc:515] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.720701 1064 consensus_queue.cc:260] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.722656 3555 leader_election.cc:290] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:24.727628 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:24.729986 792 leader_election.cc:343] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.735574 3555 consensus_queue.cc:1048] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:24.736372 666 raft_consensus.cc:2468] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:24.739949 1064 ts_tablet_manager.cc:1434] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.137s user 0.008s sys 0.000s
I20260810 07:53:24.742420 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:24.743023 891 raft_consensus.cc:2468] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:24.748075 3442 consensus_queue.cc:1048] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:24.758945 1047 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:24.762050 1064 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:24.772492 1047 ts_tablet_manager.cc:1403] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.209s user 0.016s sys 0.000s
I20260810 07:53:24.774267 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:24.783553 1047 raft_consensus.cc:359] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.728104 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:24.794528 409 raft_consensus.cc:3060] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.741966 3217 leader_election.cc:290] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.744292 308 leader_election.cc:304] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
W20260810 07:53:24.799576 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 55385.9us
I20260810 07:53:24.802999 409 raft_consensus.cc:2468] T 5eb896c8922e412c97e2cc4891298830 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:24.803675 3217 raft_consensus.cc:2804] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.805042 792 leader_election.cc:304] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:24.805882 3555 raft_consensus.cc:2804] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.806420 3555 raft_consensus.cc:697] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:24.804623 3217 raft_consensus.cc:697] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:24.807312 3555 consensus_queue.cc:237] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.807659 3217 consensus_queue.cc:237] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.812860 3558 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:24.817423 3563 raft_consensus.cc:493] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.824815 3563 raft_consensus.cc:515] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.828253 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:24.830019 309 leader_election.cc:343] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.837868 3569 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:24.845532 3569 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:24.847239 3555 raft_consensus.cc:493] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.847877 3555 raft_consensus.cc:515] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.850207 3555 leader_election.cc:290] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:24.852362 891 raft_consensus.cc:3060] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.851655 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:24.855095 1064 tablet_bootstrap.cc:654] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.856793 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:24.860495 409 raft_consensus.cc:2393] T f4a46a037cae4f599398b289bb6a2d84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:24.869426 3397 raft_consensus.cc:493] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:24.870005 3397 raft_consensus.cc:515] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.873150 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:24.871986 3397 leader_election.cc:290] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:24.874507 309 leader_election.cc:343] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.862293 3569 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:24.889402 3555 consensus_queue.cc:1048] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:24.828640 3442 consensus_queue.cc:1048] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:24.894660 1047 raft_consensus.cc:385] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:24.894968 1047 raft_consensus.cc:740] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:24.839104 3515 consensus_queue.cc:1048] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:24.851939 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a46a037cae4f599398b289bb6a2d84" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:24.860697 3367 consensus_queue.cc:1048] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:24.885219 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:24.887897 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:24.902137 3367 consensus_queue.cc:1048] T a4863f3f03104d31ad4dafda6f33c9af P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:24.895550 1047 consensus_queue.cc:260] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.908324 664 raft_consensus.cc:2468] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:24.911597 792 leader_election.cc:304] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:24.912920 3555 raft_consensus.cc:2749] T f4a46a037cae4f599398b289bb6a2d84 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:24.915330 1047 ts_tablet_manager.cc:1434] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.142s user 0.005s sys 0.001s
I20260810 07:53:24.916460 1047 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:24.916844 3563 leader_election.cc:290] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:24.923193 308 leader_election.cc:343] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:24.929347 3367 consensus_queue.cc:1048] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:24.929766 3113 raft_consensus.cc:515] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.923623 308 leader_election.cc:304] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:24.932063 308 leader_election.cc:304] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:24.932636 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:24.933379 891 raft_consensus.cc:3060] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.935021 3570 raft_consensus.cc:2749] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:24.935865 3570 raft_consensus.cc:2804] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:24.940846 3570 raft_consensus.cc:493] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:24.971704 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:24.972304 666 raft_consensus.cc:3060] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.973105 891 raft_consensus.cc:2468] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:24.975301 309 leader_election.cc:304] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:24.978804 3563 raft_consensus.cc:2804] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:24.979372 3563 raft_consensus.cc:697] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:24.980499 3563 consensus_queue.cc:237] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.991779 1047 tablet_bootstrap.cc:654] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:24.944028 3570 raft_consensus.cc:3060] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:24.993172 3113 leader_election.cc:290] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:24.996344 1071 tablet_bootstrap.cc:654] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.027876 3113 raft_consensus.cc:493] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.028810 3113 raft_consensus.cc:515] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:24.998059 666 raft_consensus.cc:2468] T e1fa9200140a41dfbd886442ae336988 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:25.039137 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:25.050562 308 leader_election.cc:343] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.054728 3570 raft_consensus.cc:515] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.058251 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:25.062739 309 leader_election.cc:343] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.063235 309 leader_election.cc:304] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.071141 3217 raft_consensus.cc:2749] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:25.079486 1064 tablet_bootstrap.cc:492] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:25.079910 1064 ts_tablet_manager.cc:1403] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.318s user 0.006s sys 0.004s
I20260810 07:53:25.082497 1064 raft_consensus.cc:359] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.083150 1064 raft_consensus.cc:385] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.083388 1064 raft_consensus.cc:740] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.083781 3113 leader_election.cc:290] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.091848 1071 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.102186 1071 ts_tablet_manager.cc:1403] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.391s user 0.014s sys 0.001s
I20260810 07:53:25.122211 1047 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:25.124002 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:25.126096 1047 ts_tablet_manager.cc:1403] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.210s user 0.011s sys 0.000s
I20260810 07:53:25.123502 3570 leader_election.cc:290] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.129009 1064 consensus_queue.cc:260] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.105520 1071 raft_consensus.cc:359] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.130198 885 raft_consensus.cc:3060] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:25.131075 1064 ts_tablet_manager.cc:1434] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.051s user 0.005s sys 0.001s
I20260810 07:53:25.132184 1064 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:25.135416 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:25.136529 1071 raft_consensus.cc:385] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.137880 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:25.138541 677 raft_consensus.cc:3060] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:25.146456 677 raft_consensus.cc:2468] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:25.148089 308 leader_election.cc:304] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:25.157981 1047 raft_consensus.cc:359] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.158279 3570 raft_consensus.cc:2804] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:25.158679 1047 raft_consensus.cc:385] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.158959 1047 raft_consensus.cc:740] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.159138 3570 raft_consensus.cc:697] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:25.160084 3570 consensus_queue.cc:237] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.164053 1071 raft_consensus.cc:740] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.171041 1071 consensus_queue.cc:260] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.175441 1071 ts_tablet_manager.cc:1434] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.072s user 0.005s sys 0.000s
I20260810 07:53:25.204586 885 raft_consensus.cc:2468] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:25.215924 1071 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:25.245229 3563 raft_consensus.cc:493] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.245867 3563 raft_consensus.cc:515] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.249395 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:25.250545 308 leader_election.cc:343] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.258917 1071 tablet_bootstrap.cc:654] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.173732 1047 consensus_queue.cc:260] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.250231 3563 leader_election.cc:290] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.349727 1071 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.351603 1071 ts_tablet_manager.cc:1403] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.136s user 0.014s sys 0.011s
I20260810 07:53:25.354391 1071 raft_consensus.cc:359] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.356593 1071 raft_consensus.cc:385] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.356958 1071 raft_consensus.cc:740] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.357713 1071 consensus_queue.cc:260] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.362876 1071 ts_tablet_manager.cc:1434] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260810 07:53:25.367681 1071 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:25.391160 1071 tablet_bootstrap.cc:654] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.318238 1047 ts_tablet_manager.cc:1434] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.192s user 0.014s sys 0.000s
I20260810 07:53:25.405514 1047 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:25.412835 3570 raft_consensus.cc:493] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.413375 3570 raft_consensus.cc:515] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.416216 3570 leader_election.cc:290] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.416107 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:25.419379 308 leader_election.cc:343] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.420945 3515 consensus_queue.cc:1048] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:25.439064 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:25.457733 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 82188.8us
I20260810 07:53:25.469924 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:25.469475 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 130863us
W20260810 07:53:25.472539 309 leader_election.cc:343] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.473006 309 leader_election.cc:304] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.479491 1047 tablet_bootstrap.cc:654] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.481635 3563 raft_consensus.cc:2749] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:25.496505 309 leader_election.cc:343] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.497107 309 leader_election.cc:304] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.498657 3563 raft_consensus.cc:2749] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:25.257987 1064 tablet_bootstrap.cc:654] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.518189 1071 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.518640 1071 ts_tablet_manager.cc:1403] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.151s user 0.008s sys 0.006s
I20260810 07:53:25.532120 3480 consensus_queue.cc:1048] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:25.546411 1071 raft_consensus.cc:359] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.547091 1071 raft_consensus.cc:385] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.547344 1071 raft_consensus.cc:740] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.547974 1071 consensus_queue.cc:260] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.570403 1071 ts_tablet_manager.cc:1434] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.051s user 0.006s sys 0.001s
I20260810 07:53:25.571383 1071 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:25.580592 3570 consensus_queue.cc:1048] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:25.589386 1071 tablet_bootstrap.cc:654] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.621997 1047 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:25.622423 1047 ts_tablet_manager.cc:1403] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.217s user 0.000s sys 0.009s
I20260810 07:53:25.629078 1064 tablet_bootstrap.cc:492] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:25.629822 1064 ts_tablet_manager.cc:1403] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.498s user 0.021s sys 0.000s
W20260810 07:53:25.631664 309 consensus_peers.cc:597] T 6db16797cabc48fe8ec82355ce3f16be P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:25.637483 1071 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.637899 1071 ts_tablet_manager.cc:1403] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.000s
I20260810 07:53:25.640257 1071 raft_consensus.cc:359] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.640928 1071 raft_consensus.cc:385] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.641185 1071 raft_consensus.cc:740] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.645421 1071 consensus_queue.cc:260] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.654412 1064 raft_consensus.cc:359] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.655679 1064 raft_consensus.cc:385] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.656005 1064 raft_consensus.cc:740] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:25.659483 574 consensus_peers.cc:597] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:25.665108 1047 raft_consensus.cc:359] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.665867 1047 raft_consensus.cc:385] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.666145 1047 raft_consensus.cc:740] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:25.676117 792 consensus_peers.cc:597] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:25.678589 3570 raft_consensus.cc:493] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.679085 3570 raft_consensus.cc:515] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.679675 3578 consensus_queue.cc:1048] T 77fa93c49cb54e5fa03034eda6fc2300 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:25.681559 3570 leader_election.cc:290] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.681752 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:25.682917 308 leader_election.cc:343] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.683208 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:25.684269 1071 ts_tablet_manager.cc:1434] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
W20260810 07:53:25.684574 309 leader_election.cc:343] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.684962 309 leader_election.cc:304] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.693490 3563 raft_consensus.cc:2749] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:25.708967 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:25.727805 1047 consensus_queue.cc:260] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.730399 1047 ts_tablet_manager.cc:1434] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.108s user 0.005s sys 0.002s
I20260810 07:53:25.741422 1071 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:25.746179 1064 consensus_queue.cc:260] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.748255 1064 ts_tablet_manager.cc:1434] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.118s user 0.008s sys 0.000s
I20260810 07:53:25.749414 1064 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:25.762686 1071 tablet_bootstrap.cc:654] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.776295 3563 raft_consensus.cc:493] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.776852 3563 raft_consensus.cc:515] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.778898 3563 leader_election.cc:290] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.781051 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:25.783246 308 leader_election.cc:343] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.800370 3563 raft_consensus.cc:493] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:25.800916 3563 raft_consensus.cc:515] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.802906 3563 leader_election.cc:290] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:25.805003 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:25.805258 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:25.806699 308 leader_election.cc:343] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:25.808789 309 leader_election.cc:343] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.809230 309 leader_election.cc:304] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.810726 3563 raft_consensus.cc:2749] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:25.823855 1047 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:25.826560 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:25.833153 309 leader_election.cc:343] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:25.833670 309 leader_election.cc:304] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:25.836522 3570 raft_consensus.cc:2749] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:25.845813 3620 consensus_queue.cc:1048] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:25.870075 1071 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.870553 1071 ts_tablet_manager.cc:1403] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.129s user 0.006s sys 0.003s
I20260810 07:53:25.870661 1064 tablet_bootstrap.cc:654] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.886794 1047 tablet_bootstrap.cc:654] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.877239 1071 raft_consensus.cc:359] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:25.889014 1071 raft_consensus.cc:385] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.889283 1071 raft_consensus.cc:740] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.900354 1071 consensus_queue.cc:260] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:25.936265 32452 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet f4a46a037cae4f599398b289bb6a2d84 (table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f4a46a037cae4f599398b289bb6a2d84 consensus state has no leader
I20260810 07:53:25.937094 1071 ts_tablet_manager.cc:1434] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.066s user 0.008s sys 0.000s
I20260810 07:53:25.952910 1071 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:25.958703 1071 tablet_bootstrap.cc:654] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:25.970427 3570 consensus_queue.cc:1048] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:25.975212 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.514s (SENT)
I20260810 07:53:25.975824 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:25.985541 1071 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:25.985983 1071 ts_tablet_manager.cc:1403] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260810 07:53:25.988521 1071 raft_consensus.cc:359] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:25.990010 1071 raft_consensus.cc:385] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:25.990278 1071 raft_consensus.cc:740] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:25.990900 1071 consensus_queue.cc:260] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.001287 1047 tablet_bootstrap.cc:492] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:26.001778 1047 ts_tablet_manager.cc:1403] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.178s user 0.010s sys 0.000s
I20260810 07:53:26.005040 1071 ts_tablet_manager.cc:1434] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260810 07:53:26.006042 1071 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:26.022261 3648 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:26.023608 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 48580.2us
I20260810 07:53:26.032187 1064 tablet_bootstrap.cc:492] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
W20260810 07:53:26.040273 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.566s (SENT)
I20260810 07:53:26.040818 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:26.033694 1071 tablet_bootstrap.cc:654] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:26.049525 3650 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:26.032686 1064 ts_tablet_manager.cc:1403] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.283s user 0.008s sys 0.003s
I20260810 07:53:26.004340 1047 raft_consensus.cc:359] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.057117 1047 raft_consensus.cc:385] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.057408 1047 raft_consensus.cc:740] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.057341 1064 raft_consensus.cc:359] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.057966 1064 raft_consensus.cc:385] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.058229 1064 raft_consensus.cc:740] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.058065 1047 consensus_queue.cc:260] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.058835 1064 consensus_queue.cc:260] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.060088 1047 ts_tablet_manager.cc:1434] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.058s user 0.004s sys 0.000s
I20260810 07:53:26.060307 1064 ts_tablet_manager.cc:1434] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260810 07:53:26.080003 3563 consensus_queue.cc:1048] T e1fa9200140a41dfbd886442ae336988 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:26.092746 1047 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:26.099754 3570 consensus_queue.cc:1048] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:26.100143 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:26.116658 3480 raft_consensus.cc:493] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.117223 3480 raft_consensus.cc:515] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.119374 3480 leader_election.cc:290] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.122721 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:26.123401 409 raft_consensus.cc:2468] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:26.124944 573 leader_election.cc:304] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.126001 3367 raft_consensus.cc:2804] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:26.126464 3367 raft_consensus.cc:493] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:26.126783 3367 raft_consensus.cc:3060] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.131181 1071 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:26.131709 1071 ts_tablet_manager.cc:1403] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.126s user 0.011s sys 0.000s
I20260810 07:53:26.134436 1071 raft_consensus.cc:359] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.141465 1071 raft_consensus.cc:385] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.141824 1071 raft_consensus.cc:740] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.142524 1071 consensus_queue.cc:260] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.149871 1071 ts_tablet_manager.cc:1434] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260810 07:53:26.165253 1071 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:26.171821 1047 tablet_bootstrap.cc:654] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:26.160081 3367 raft_consensus.cc:515] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.231276 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.231940 888 raft_consensus.cc:2468] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:26.236953 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:26.237568 409 raft_consensus.cc:3060] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.247181 1064 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:26.254053 1071 tablet_bootstrap.cc:654] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:26.267966 1047 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:26.287276 409 raft_consensus.cc:2468] T fe8b4892a53742f48dd43397c4f8e88f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:26.289059 3652 raft_consensus.cc:493] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.289539 3652 raft_consensus.cc:515] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.291496 3652 leader_election.cc:290] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:26.300607 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 60374.5us
I20260810 07:53:26.302424 3570 consensus_queue.cc:1048] T a4d1c8d270f744ee9849475c9128414b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:26.305680 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:26.318109 309 leader_election.cc:343] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:26.281031 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 45334.6us
I20260810 07:53:26.288790 573 leader_election.cc:304] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.293176 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:26.326732 3515 raft_consensus.cc:2804] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:26.327309 3515 raft_consensus.cc:697] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:26.328593 1071 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:26.328245 3515 consensus_queue.cc:237] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.329167 1071 ts_tablet_manager.cc:1403] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.164s user 0.010s sys 0.003s
I20260810 07:53:26.319147 3660 raft_consensus.cc:493] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.329859 3660 raft_consensus.cc:515] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.332382 1071 raft_consensus.cc:359] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.333173 1071 raft_consensus.cc:385] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.333447 1071 raft_consensus.cc:740] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
W20260810 07:53:26.334162 308 leader_election.cc:343] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.334581 308 leader_election.cc:304] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:26.334772 1047 ts_tablet_manager.cc:1403] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.242s user 0.008s sys 0.005s
I20260810 07:53:26.338384 3563 raft_consensus.cc:2749] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.340091 3367 leader_election.cc:290] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.355976 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:26.370934 3660 leader_election.cc:290] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.372886 1047 raft_consensus.cc:359] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.373628 1047 raft_consensus.cc:385] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.373916 1047 raft_consensus.cc:740] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.374891 1071 consensus_queue.cc:260] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.380201 1071 ts_tablet_manager.cc:1434] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
I20260810 07:53:26.407219 1064 tablet_bootstrap.cc:654] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:26.407629 3113 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.408077 3113 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.410444 3113 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.418125 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.374538 1047 consensus_queue.cc:260] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:26.419865 309 leader_election.cc:343] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.429678 3570 raft_consensus.cc:493] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.430248 3570 raft_consensus.cc:515] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.432322 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:26.436403 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:26.437511 308 leader_election.cc:343] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.437908 308 leader_election.cc:304] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:26.438601 664 raft_consensus.cc:3060] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:26.439229 308 leader_election.cc:343] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.441838 3583 raft_consensus.cc:2749] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.443228 3570 leader_election.cc:290] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.445406 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:26.447465 309 leader_election.cc:343] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.447928 309 leader_election.cc:304] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:26.449585 1071 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:26.454109 3583 raft_consensus.cc:2749] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.461681 1071 tablet_bootstrap.cc:654] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:26.470748 1047 ts_tablet_manager.cc:1434] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.135s user 0.006s sys 0.000s
W20260810 07:53:26.474884 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 141403us
I20260810 07:53:26.479758 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:26.481945 895 raft_consensus.cc:3060] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.481197 883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eb896c8922e412c97e2cc4891298830" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.524456 1047 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:26.541178 574 consensus_peers.cc:597] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef. This is attempt 6: this message will repeat every 5th retry.
I20260810 07:53:26.569669 1064 tablet_bootstrap.cc:492] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:26.570303 1064 ts_tablet_manager.cc:1403] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.323s user 0.006s sys 0.010s
I20260810 07:53:26.575315 3660 raft_consensus.cc:493] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.576880 3660 raft_consensus.cc:515] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.554203 895 raft_consensus.cc:2468] T fe8b4892a53742f48dd43397c4f8e88f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:26.583271 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:26.584852 3660 leader_election.cc:290] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.585335 409 raft_consensus.cc:2468] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:26.587891 3650 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:26.588552 3650 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:26.587235 573 leader_election.cc:304] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.593647 3660 raft_consensus.cc:2804] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:26.594115 3660 raft_consensus.cc:493] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:26.594527 3660 raft_consensus.cc:3060] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.597023 3650 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:26.601233 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:26.604566 574 leader_election.cc:304] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:26.605422 3480 raft_consensus.cc:2749] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.618081 1064 raft_consensus.cc:359] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.619024 1064 raft_consensus.cc:385] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.619679 1064 raft_consensus.cc:740] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:26.645525 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:26.656033 1047 tablet_bootstrap.cc:654] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:26.666445 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 57699.8us
I20260810 07:53:26.670776 883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.675499 1071 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:26.675933 1071 ts_tablet_manager.cc:1403] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.227s user 0.008s sys 0.012s
I20260810 07:53:26.679409 1071 raft_consensus.cc:359] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.680029 1071 raft_consensus.cc:385] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.680321 1071 raft_consensus.cc:740] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.625293 1064 consensus_queue.cc:260] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.687279 3620 raft_consensus.cc:493] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.687752 3620 raft_consensus.cc:515] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.692159 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:26.693006 3620 leader_election.cc:290] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:26.709323 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db16797cabc48fe8ec82355ce3f16be" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:26.710981 792 leader_election.cc:304] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:26.711869 3583 raft_consensus.cc:493] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.712343 3583 raft_consensus.cc:515] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.715799 883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.716355 3583 leader_election.cc:290] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:26.717612 309 leader_election.cc:343] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.716979 3620 raft_consensus.cc:2749] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.722473 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:26.723822 308 leader_election.cc:343] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.724226 308 leader_election.cc:304] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:26.725544 3583 raft_consensus.cc:2749] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:26.730446 1064 ts_tablet_manager.cc:1434] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.160s user 0.008s sys 0.001s
I20260810 07:53:26.736769 1071 consensus_queue.cc:260] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.738483 1071 ts_tablet_manager.cc:1434] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.062s user 0.007s sys 0.000s
I20260810 07:53:26.739861 3669 raft_consensus.cc:493] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:26.740339 3669 raft_consensus.cc:515] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.742554 3669 leader_election.cc:290] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.744524 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:26.745368 409 raft_consensus.cc:2468] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:26.757627 573 leader_election.cc:304] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.758486 3669 raft_consensus.cc:2804] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:26.758916 3669 raft_consensus.cc:493] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:26.759335 3669 raft_consensus.cc:3060] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.761353 1064 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:26.801949 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 87088.3us
W20260810 07:53:26.802634 574 leader_election.cc:343] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.803640 3660 raft_consensus.cc:515] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.816859 3660 leader_election.cc:290] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.823522 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:26.824188 409 raft_consensus.cc:3060] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.833117 1071 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:26.836544 3669 raft_consensus.cc:515] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.840204 409 raft_consensus.cc:2468] T 0d3aaf0099934d1e9bede4687b36ce44 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:26.846061 1047 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:26.846663 1047 ts_tablet_manager.cc:1403] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.335s user 0.006s sys 0.009s
I20260810 07:53:26.849517 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:26.849861 573 leader_election.cc:304] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.850104 414 raft_consensus.cc:3060] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.851107 3677 raft_consensus.cc:2804] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:26.851536 3677 raft_consensus.cc:697] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:26.852579 3677 consensus_queue.cc:237] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.853487 3680 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:26.854036 3680 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:26.857296 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:26.860615 414 raft_consensus.cc:2468] T 8ea09585009e4a3ea33f620182bfee60 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:26.862874 573 leader_election.cc:304] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:26.864146 3367 raft_consensus.cc:2804] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:26.864743 3367 raft_consensus.cc:697] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:26.865792 3367 consensus_queue.cc:237] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:26.866547 1047 raft_consensus.cc:359] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.867901 1047 raft_consensus.cc:385] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:26.869760 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:26.871268 892 raft_consensus.cc:3060] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:26.871367 574 leader_election.cc:343] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.868502 1047 raft_consensus.cc:740] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:26.871817 894 raft_consensus.cc:3060] T 849197942f134542a387a10167fc7481 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:26.873505 1047 consensus_queue.cc:260] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:26.880123 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:26.882932 1047 ts_tablet_manager.cc:1434] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.036s user 0.009s sys 0.001s
I20260810 07:53:26.886188 1047 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:26.887754 574 leader_election.cc:343] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.921053 3680 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:26.928484 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea09585009e4a3ea33f620182bfee60" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
W20260810 07:53:26.937108 574 leader_election.cc:343] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:26.918329 3669 leader_election.cc:290] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:26.965389 1047 tablet_bootstrap.cc:654] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:27.015000 3685 log.cc:927] Time spent T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.055s user 0.000s sys 0.002s
I20260810 07:53:27.053467 1047 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:27.054071 1047 ts_tablet_manager.cc:1403] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.169s user 0.014s sys 0.008s
W20260810 07:53:27.056677 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:27.077265 1071 tablet_bootstrap.cc:654] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.085340 1064 tablet_bootstrap.cc:654] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.142184 1071 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:27.143036 1071 ts_tablet_manager.cc:1403] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.310s user 0.008s sys 0.008s
I20260810 07:53:27.148082 1047 raft_consensus.cc:359] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.148792 1047 raft_consensus.cc:385] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.149050 1047 raft_consensus.cc:740] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.168568 1071 raft_consensus.cc:359] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:27.169056 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 20a9ab67614949bb93925144c4675f01 (table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 20a9ab67614949bb93925144c4675f01 consensus state has no leader
I20260810 07:53:27.175673 1047 consensus_queue.cc:260] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.169349 1071 raft_consensus.cc:385] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.196683 1071 raft_consensus.cc:740] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.197461 1071 consensus_queue.cc:260] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.205790 1047 ts_tablet_manager.cc:1434] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.151s user 0.009s sys 0.001s
I20260810 07:53:27.213816 1071 ts_tablet_manager.cc:1434] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.070s user 0.005s sys 0.003s
I20260810 07:53:27.254671 1064 tablet_bootstrap.cc:492] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:27.255141 1064 ts_tablet_manager.cc:1403] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.494s user 0.003s sys 0.013s
I20260810 07:53:27.259446 1071 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:27.267107 1071 tablet_bootstrap.cc:654] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:27.277102 2560 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.680s (SENT) [suppressed 2 similar messages]
I20260810 07:53:27.318427 3672 consensus_queue.cc:1048] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:27.327412 1071 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:27.327909 1071 ts_tablet_manager.cc:1403] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.069s user 0.010s sys 0.000s
I20260810 07:53:27.359139 1071 raft_consensus.cc:359] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.360087 1071 raft_consensus.cc:385] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.360416 1071 raft_consensus.cc:740] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.361577 1071 consensus_queue.cc:260] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.367591 1047 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:27.368366 1071 ts_tablet_manager.cc:1434] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.040s user 0.004s sys 0.003s
I20260810 07:53:27.381388 3669 consensus_queue.cc:1048] T fe8b4892a53742f48dd43397c4f8e88f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:27.392000 3583 raft_consensus.cc:493] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:27.392508 3583 raft_consensus.cc:515] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.394570 3583 leader_election.cc:290] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:27.397746 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:27.399592 309 leader_election.cc:343] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:27.399967 1047 tablet_bootstrap.cc:654] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.403728 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:27.408460 1071 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:27.413462 3583 raft_consensus.cc:493] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:27.414557 3583 raft_consensus.cc:515] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.307543 1064 raft_consensus.cc:359] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.419284 1064 raft_consensus.cc:385] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.419589 1064 raft_consensus.cc:740] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:27.421808 3244 log.cc:927] Time spent T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531: Append to log took a long time: real 0.096s user 0.003s sys 0.000s
I20260810 07:53:27.423259 3583 leader_election.cc:290] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:27.431192 1064 consensus_queue.cc:260] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.433149 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:27.434648 309 leader_election.cc:343] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:27.440042 1064 ts_tablet_manager.cc:1434] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.185s user 0.010s sys 0.000s
I20260810 07:53:27.441097 1064 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:27.447147 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 48091.1us
I20260810 07:53:27.450151 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:27.451570 308 leader_election.cc:343] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:27.451961 308 leader_election.cc:304] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:27.452615 3113 raft_consensus.cc:2749] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:27.455273 308 leader_election.cc:343] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:27.455644 308 leader_election.cc:304] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:27.456980 1047 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:27.457460 1047 ts_tablet_manager.cc:1403] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.090s user 0.004s sys 0.009s
I20260810 07:53:27.457615 3113 raft_consensus.cc:2749] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:27.488875 1071 tablet_bootstrap.cc:654] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.536414 1064 tablet_bootstrap.cc:654] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.561379 1071 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:27.561861 1071 ts_tablet_manager.cc:1403] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.155s user 0.007s sys 0.007s
W20260810 07:53:27.565701 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 49115.6us
I20260810 07:53:27.565214 1071 raft_consensus.cc:359] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.566373 1071 raft_consensus.cc:385] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.566654 1071 raft_consensus.cc:740] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.567688 1071 consensus_queue.cc:260] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.591867 3705 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:27.592572 3705 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:27.595830 3705 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:27.597563 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
W20260810 07:53:27.599874 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:27.600250 792 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:27.600023 1047 raft_consensus.cc:359] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.601008 3705 raft_consensus.cc:2749] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:27.606640 1047 raft_consensus.cc:385] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.607053 1047 raft_consensus.cc:740] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.608119 1047 consensus_queue.cc:260] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.610309 1071 ts_tablet_manager.cc:1434] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.048s user 0.001s sys 0.007s
I20260810 07:53:27.611333 1071 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:27.658700 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:27.659780 1071 tablet_bootstrap.cc:654] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.694868 892 raft_consensus.cc:3060] T 8ea09585009e4a3ea33f620182bfee60 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:27.704011 3669 raft_consensus.cc:493] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:27.704826 3669 raft_consensus.cc:515] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:27.713233 1047 ts_tablet_manager.cc:1434] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.255s user 0.007s sys 0.000s
I20260810 07:53:27.725263 1071 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:27.725695 1071 ts_tablet_manager.cc:1403] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.115s user 0.012s sys 0.004s
I20260810 07:53:27.731120 1071 raft_consensus.cc:359] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.731814 1071 raft_consensus.cc:385] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.732088 1071 raft_consensus.cc:740] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.732867 1071 consensus_queue.cc:260] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.736938 3677 consensus_queue.cc:1048] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:27.738317 1071 ts_tablet_manager.cc:1434] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260810 07:53:27.754773 1071 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:27.774004 1071 tablet_bootstrap.cc:654] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:27.790329 1047 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:27.819068 3669 leader_election.cc:290] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:27.819211 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:27.819859 414 raft_consensus.cc:2468] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:27.821231 573 leader_election.cc:304] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:27.822467 3677 raft_consensus.cc:2804] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:27.822932 3677 raft_consensus.cc:493] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:27.823335 3677 raft_consensus.cc:3060] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:27.843295 574 consensus_peers.cc:597] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:27.887617 1064 tablet_bootstrap.cc:492] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:27.888088 1064 ts_tablet_manager.cc:1403] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.447s user 0.004s sys 0.009s
I20260810 07:53:27.890956 1064 raft_consensus.cc:359] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.891620 1064 raft_consensus.cc:385] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:27.891891 1064 raft_consensus.cc:740] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:27.892469 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:27.896926 574 leader_election.cc:343] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:27.928411 3669 consensus_queue.cc:1048] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:27.938906 1064 consensus_queue.cc:260] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.940570 1064 ts_tablet_manager.cc:1434] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.052s user 0.007s sys 0.001s
I20260810 07:53:27.941542 1064 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:27.948779 1071 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:27.949298 1071 ts_tablet_manager.cc:1403] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.195s user 0.016s sys 0.048s
I20260810 07:53:27.984638 1047 tablet_bootstrap.cc:654] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:27.986184 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.680s (SENT)
I20260810 07:53:27.986860 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:27.988940 3722 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:27.991581 3583 raft_consensus.cc:493] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:27.992283 3583 raft_consensus.cc:515] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:27.997263 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:27.999505 309 leader_election.cc:343] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.000172 3583 leader_election.cc:290] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:28.036089 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:28.042285 308 leader_election.cc:343] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.042553 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:28.042840 308 leader_election.cc:304] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:28.047976 3583 raft_consensus.cc:2749] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:28.008170 1071 raft_consensus.cc:359] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.053433 1071 raft_consensus.cc:385] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.053725 1071 raft_consensus.cc:740] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:28.054352 1071 consensus_queue.cc:260] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.055938 1071 ts_tablet_manager.cc:1434] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.106s user 0.005s sys 0.000s
I20260810 07:53:28.089619 3669 consensus_queue.cc:1048] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:28.114683 1064 tablet_bootstrap.cc:654] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:28.122131 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.581s (SENT)
I20260810 07:53:28.122591 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:28.124025 1071 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:28.159722 3721 raft_consensus.cc:493] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.160554 3721 raft_consensus.cc:515] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.161264 3677 raft_consensus.cc:515] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.165112 3515 raft_consensus.cc:493] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.165745 3515 raft_consensus.cc:515] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.166679 3721 leader_election.cc:290] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:28.169986 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 47901.1us
I20260810 07:53:28.169978 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:28.170655 411 raft_consensus.cc:2468] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:28.168504 3515 leader_election.cc:290] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:28.172503 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:28.174890 573 leader_election.cc:304] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:28.175727 3515 raft_consensus.cc:2804] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:28.176293 3515 raft_consensus.cc:493] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:28.176970 3515 raft_consensus.cc:3060] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.178143 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:28.181130 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:28.182484 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:28.182896 574 leader_election.cc:343] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.183802 574 leader_election.cc:343] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.184079 309 leader_election.cc:343] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.169998 3725 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:28.194376 3583 raft_consensus.cc:493] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.195070 3583 raft_consensus.cc:515] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.198087 1071 tablet_bootstrap.cc:654] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:28.228228 3583 leader_election.cc:290] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:28.230252 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:28.232088 309 leader_election.cc:343] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.245141 308 leader_election.cc:343] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.247419 308 leader_election.cc:304] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:28.249023 1047 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:28.249722 1047 ts_tablet_manager.cc:1403] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.459s user 0.014s sys 0.000s
I20260810 07:53:28.250237 3583 raft_consensus.cc:2749] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:28.260594 1064 tablet_bootstrap.cc:492] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:28.261114 1064 ts_tablet_manager.cc:1403] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.320s user 0.009s sys 0.003s
I20260810 07:53:28.264036 1064 raft_consensus.cc:359] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.290338 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:28.264670 1064 raft_consensus.cc:385] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.302903 1064 raft_consensus.cc:740] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:28.303762 1064 consensus_queue.cc:260] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.305238 3515 raft_consensus.cc:515] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.326982 3677 leader_election.cc:290] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:28.341823 1047 raft_consensus.cc:359] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.342854 1047 raft_consensus.cc:385] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.343181 1047 raft_consensus.cc:740] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:28.343959 1047 consensus_queue.cc:260] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.361414 1064 ts_tablet_manager.cc:1434] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.100s user 0.007s sys 0.000s
I20260810 07:53:28.397600 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:28.398907 1064 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:28.424199 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:28.424794 412 raft_consensus.cc:3060] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.432126 412 raft_consensus.cc:2468] T 2a91af48fe6a4bd986b5266f09fad6b2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:28.432674 1047 ts_tablet_manager.cc:1434] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.183s user 0.006s sys 0.000s
I20260810 07:53:28.433811 1047 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:28.433739 1071 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:28.434782 1071 ts_tablet_manager.cc:1403] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.311s user 0.013s sys 0.001s
I20260810 07:53:28.439538 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:28.439844 1071 raft_consensus.cc:359] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.440122 411 raft_consensus.cc:3060] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.397488 3515 leader_election.cc:290] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:28.399470 574 leader_election.cc:343] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.488384 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 88941.4us
I20260810 07:53:28.493585 573 leader_election.cc:304] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:28.497116 3480 raft_consensus.cc:2804] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:28.497628 3480 raft_consensus.cc:697] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:28.498664 3480 consensus_queue.cc:237] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.511307 1064 tablet_bootstrap.cc:654] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:28.512782 308 leader_election.cc:343] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.514297 308 leader_election.cc:304] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:28.515027 3720 raft_consensus.cc:2749] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:28.516822 1071 raft_consensus.cc:385] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.517156 1071 raft_consensus.cc:740] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:28.517870 1071 consensus_queue.cc:260] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.533074 411 raft_consensus.cc:2468] T 69f1405e9526433a8fd727ed237f8c84 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:28.535573 573 leader_election.cc:304] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:28.537257 3515 raft_consensus.cc:2804] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:28.537946 3515 raft_consensus.cc:697] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:28.559760 3515 consensus_queue.cc:237] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.574602 3480 raft_consensus.cc:493] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.575392 3480 raft_consensus.cc:515] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:28.579795 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:28.603468 3480 leader_election.cc:290] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:28.603403 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:28.604004 411 raft_consensus.cc:2468] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:28.605304 573 leader_election.cc:304] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:28.606264 3480 raft_consensus.cc:2804] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:28.606762 3480 raft_consensus.cc:493] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:28.606707 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:28.607322 3480 raft_consensus.cc:3060] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.607721 897 raft_consensus.cc:2468] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:28.610986 1071 ts_tablet_manager.cc:1434] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.176s user 0.003s sys 0.004s
I20260810 07:53:28.617794 3660 raft_consensus.cc:493] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.618347 3660 raft_consensus.cc:515] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.620388 3660 leader_election.cc:290] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:28.622303 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:28.622855 411 raft_consensus.cc:2468] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:28.624055 573 leader_election.cc:304] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:28.624984 3677 raft_consensus.cc:2804] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:28.625308 3677 raft_consensus.cc:493] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:28.625597 3677 raft_consensus.cc:3060] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.630854 3692 raft_consensus.cc:493] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.631395 3692 raft_consensus.cc:515] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.633649 3692 leader_election.cc:290] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:28.633592 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:28.635540 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:28.637526 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d3aaf0099934d1e9bede4687b36ce44" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:28.638880 3677 raft_consensus.cc:515] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:28.638932 574 leader_election.cc:343] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.644089 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:28.641482 3480 raft_consensus.cc:515] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.644968 411 raft_consensus.cc:3060] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.642525 3677 leader_election.cc:290] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:28.649523 1071 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:28.649521 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7084712ff04ac48a54171534e04d69" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:28.651019 792 leader_election.cc:304] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:28.651813 3692 raft_consensus.cc:2749] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:28.653671 574 leader_election.cc:343] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.657653 411 raft_consensus.cc:2468] T 0d7084712ff04ac48a54171534e04d69 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:28.659646 573 leader_election.cc:304] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters: 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:28.661937 3515 raft_consensus.cc:2804] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:28.662377 3515 raft_consensus.cc:697] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:28.663174 3515 consensus_queue.cc:237] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.674875 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:28.675056 3480 leader_election.cc:290] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:28.675436 412 raft_consensus.cc:3060] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.687494 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47792551bb243b7b3f6de904c17c47b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:28.688117 897 raft_consensus.cc:3060] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:28.699887 897 raft_consensus.cc:2468] T c47792551bb243b7b3f6de904c17c47b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:28.704435 1064 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:28.704923 1064 ts_tablet_manager.cc:1403] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.306s user 0.011s sys 0.004s
I20260810 07:53:28.706137 574 leader_election.cc:304] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:28.706985 3677 raft_consensus.cc:2804] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:28.707551 3677 raft_consensus.cc:697] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:28.707479 1064 raft_consensus.cc:359] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.708182 1064 raft_consensus.cc:385] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.708485 1064 raft_consensus.cc:740] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:28.708599 3677 consensus_queue.cc:237] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.764261 1064 consensus_queue.cc:260] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.774216 1064 ts_tablet_manager.cc:1434] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.069s user 0.007s sys 0.000s
I20260810 07:53:28.778571 1047 tablet_bootstrap.cc:654] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:28.783458 1071 tablet_bootstrap.cc:654] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:28.785508 412 raft_consensus.cc:2468] T c47792551bb243b7b3f6de904c17c47b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:28.834019 3721 raft_consensus.cc:493] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:28.838276 3721 raft_consensus.cc:515] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:28.845232 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:28.849186 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:28.853047 309 leader_election.cc:343] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:28.854876 308 leader_election.cc:343] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:28.855245 308 leader_election.cc:304] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:28.862039 3744 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:28.870950 3744 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:28.883525 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:28.889020 3702 raft_consensus.cc:2749] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:28.855182 3721 leader_election.cc:290] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:28.895131 1064 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:28.902227 1064 tablet_bootstrap.cc:654] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:28.911769 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9b4339a26ad540f4b385f69fd0dcdc5d consensus state has no leader
I20260810 07:53:28.918970 3744 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:28.930328 1047 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:28.930760 1047 ts_tablet_manager.cc:1403] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.497s user 0.007s sys 0.004s
W20260810 07:53:28.951085 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 75316.5us
W20260810 07:53:28.953097 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 78698.9us
I20260810 07:53:28.955415 1047 raft_consensus.cc:359] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:28.956110 1047 raft_consensus.cc:385] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:28.956400 1047 raft_consensus.cc:740] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:28.988963 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:28.996642 897 raft_consensus.cc:3060] T 0d3aaf0099934d1e9bede4687b36ce44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:29.013568 1047 consensus_queue.cc:260] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.014324 1064 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.014752 1064 ts_tablet_manager.cc:1403] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.120s user 0.022s sys 0.011s
I20260810 07:53:29.017428 1064 raft_consensus.cc:359] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.018055 1064 raft_consensus.cc:385] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.018299 1064 raft_consensus.cc:740] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.018923 1064 consensus_queue.cc:260] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.028474 1047 ts_tablet_manager.cc:1434] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.097s user 0.007s sys 0.000s
I20260810 07:53:29.053191 1064 ts_tablet_manager.cc:1434] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.038s user 0.008s sys 0.000s
I20260810 07:53:29.054191 1064 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.091037 3702 raft_consensus.cc:493] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.091897 3702 raft_consensus.cc:515] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.094456 3702 leader_election.cc:290] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:29.095311 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:29.096768 308 leader_election.cc:343] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:29.104321 1071 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:29.109082 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:29.139912 1064 tablet_bootstrap.cc:654] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:29.140024 309 leader_election.cc:343] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:29.146193 309 leader_election.cc:304] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:29.169010 3702 raft_consensus.cc:2749] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:29.207634 1071 ts_tablet_manager.cc:1403] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.558s user 0.001s sys 0.016s
I20260810 07:53:29.236966 1064 tablet_bootstrap.cc:492] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.237464 1064 ts_tablet_manager.cc:1403] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.183s user 0.007s sys 0.005s
I20260810 07:53:29.240172 1064 raft_consensus.cc:359] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.250267 1064 raft_consensus.cc:385] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.250682 1064 raft_consensus.cc:740] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.277056 1064 consensus_queue.cc:260] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.279515 1064 ts_tablet_manager.cc:1434] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260810 07:53:29.280537 1064 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.237138 1071 raft_consensus.cc:359] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.282694 1071 raft_consensus.cc:385] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.282990 1071 raft_consensus.cc:740] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.313033 1071 consensus_queue.cc:260] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.314843 1071 ts_tablet_manager.cc:1434] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.107s user 0.006s sys 0.000s
I20260810 07:53:29.332907 1064 tablet_bootstrap.cc:654] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.346562 1047 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:29.358413 1071 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:29.361913 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56722.8us
W20260810 07:53:29.377923 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:29.394833 1064 tablet_bootstrap.cc:492] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.395481 1064 ts_tablet_manager.cc:1403] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.003s
I20260810 07:53:29.352295 1047 tablet_bootstrap.cc:654] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.405393 1064 raft_consensus.cc:359] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.406139 1064 raft_consensus.cc:385] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.406430 1064 raft_consensus.cc:740] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.409022 1064 consensus_queue.cc:260] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.421396 1064 ts_tablet_manager.cc:1434] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260810 07:53:29.437572 1064 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.444054 3766 raft_consensus.cc:493] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.444559 3766 raft_consensus.cc:515] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.446439 3766 leader_election.cc:290] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:29.462679 1064 tablet_bootstrap.cc:654] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.469444 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:29.470934 1047 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:29.471349 1047 ts_tablet_manager.cc:1403] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.125s user 0.009s sys 0.001s
I20260810 07:53:29.474833 1047 raft_consensus.cc:359] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.475487 1047 raft_consensus.cc:385] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.475741 1047 raft_consensus.cc:740] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.476464 1047 consensus_queue.cc:260] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.489395 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:29.497180 412 raft_consensus.cc:2468] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:29.492579 1071 tablet_bootstrap.cc:654] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:29.527163 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 49903.7us
I20260810 07:53:29.529100 1047 ts_tablet_manager.cc:1434] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.057s user 0.007s sys 0.000s
I20260810 07:53:29.531066 3787 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:29.531548 3787 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:29.542271 1064 tablet_bootstrap.cc:492] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.543339 1064 ts_tablet_manager.cc:1403] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.008s
I20260810 07:53:29.550033 1047 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:29.551647 1064 raft_consensus.cc:359] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.552389 1064 raft_consensus.cc:385] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.552695 1064 raft_consensus.cc:740] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.553390 1064 consensus_queue.cc:260] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.554879 1064 ts_tablet_manager.cc:1434] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260810 07:53:29.555886 1064 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.573637 3677 consensus_queue.cc:1048] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:29.574240 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
W20260810 07:53:29.577065 32563 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.654s (SENT) [suppressed 2 similar messages]
I20260810 07:53:29.581113 3787 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:29.582087 3766 raft_consensus.cc:493] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.582654 3766 raft_consensus.cc:515] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.592398 1047 tablet_bootstrap.cc:654] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.595774 3766 leader_election.cc:290] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:29.603227 1064 tablet_bootstrap.cc:654] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.662420 1064 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.662873 1064 ts_tablet_manager.cc:1403] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.107s user 0.010s sys 0.000s
W20260810 07:53:29.669775 2816 debug-util.cc:398] Leaking SignalData structure 0x7b080006f040 after lost signal to thread 1539
W20260810 07:53:29.670600 2816 debug-util.cc:398] Leaking SignalData structure 0x7b080007c7e0 after lost signal to thread 2825
W20260810 07:53:29.671880 792 leader_election.cc:343] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:29.682386 1064 raft_consensus.cc:359] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.683166 1064 raft_consensus.cc:385] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.683321 1071 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:29.683460 1064 raft_consensus.cc:740] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.683908 1071 ts_tablet_manager.cc:1403] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.326s user 0.017s sys 0.014s
I20260810 07:53:29.684165 1064 consensus_queue.cc:260] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.699149 3751 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.700198 1071 raft_consensus.cc:359] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.712872 1071 raft_consensus.cc:385] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.713294 1071 raft_consensus.cc:740] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.699740 3751 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.714558 1071 consensus_queue.cc:260] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:29.717454 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:29.715849 3751 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:29.719134 309 leader_election.cc:343] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:29.717679 1064 ts_tablet_manager.cc:1434] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.055s user 0.008s sys 0.000s
I20260810 07:53:29.726328 1071 ts_tablet_manager.cc:1434] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260810 07:53:29.729595 1064 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.742941 1064 tablet_bootstrap.cc:654] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.764799 3515 consensus_queue.cc:1048] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:29.785038 1047 tablet_bootstrap.cc:492] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:29.785475 1047 ts_tablet_manager.cc:1403] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.236s user 0.023s sys 0.011s
I20260810 07:53:29.787938 1047 raft_consensus.cc:359] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.788575 1047 raft_consensus.cc:385] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.790150 1047 raft_consensus.cc:740] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.790977 1064 tablet_bootstrap.cc:492] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:29.791375 1064 ts_tablet_manager.cc:1403] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.004s
I20260810 07:53:29.794502 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:29.795416 412 raft_consensus.cc:2468] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:29.804505 1047 consensus_queue.cc:260] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.806077 1047 ts_tablet_manager.cc:1434] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260810 07:53:29.827662 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:29.828272 677 raft_consensus.cc:2468] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:29.833165 1047 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:29.833462 1064 raft_consensus.cc:359] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.834221 1064 raft_consensus.cc:385] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:29.834534 1064 raft_consensus.cc:740] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:29.835033 792 leader_election.cc:304] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:29.835822 3766 raft_consensus.cc:2804] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:29.836288 3766 raft_consensus.cc:493] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:29.835229 1064 consensus_queue.cc:260] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.836661 3766 raft_consensus.cc:3060] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:29.834549 3776 raft_consensus.cc:493] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.841233 3776 raft_consensus.cc:515] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.843159 3776 leader_election.cc:290] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:29.847460 3766 raft_consensus.cc:515] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.849483 3766 leader_election.cc:290] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:29.860632 1047 tablet_bootstrap.cc:654] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:29.872964 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 295901us
I20260810 07:53:29.894629 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:29.913180 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:29.954102 3677 raft_consensus.cc:493] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:29.954633 3677 raft_consensus.cc:515] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:29.966873 3515 consensus_queue.cc:1048] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:29.888770 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 53808.8us
I20260810 07:53:29.906301 1064 ts_tablet_manager.cc:1434] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.115s user 0.013s sys 0.000s
I20260810 07:53:29.978502 1064 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:29.980150 792 leader_election.cc:304] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:29.990604 1047 tablet_bootstrap.cc:492] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:29.990741 3776 raft_consensus.cc:2804] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:29.991079 3776 raft_consensus.cc:493] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:29.991195 1047 ts_tablet_manager.cc:1403] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.158s user 0.022s sys 0.003s
I20260810 07:53:29.991405 3776 raft_consensus.cc:3060] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:29.994059 1064 tablet_bootstrap.cc:654] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:29.993873 1047 raft_consensus.cc:359] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.001214 1047 raft_consensus.cc:385] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.001631 1047 raft_consensus.cc:740] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.002403 1047 consensus_queue.cc:260] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.003456 3776 raft_consensus.cc:515] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.004289 1047 ts_tablet_manager.cc:1434] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260810 07:53:30.015009 886 raft_consensus.cc:3060] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.015678 1064 tablet_bootstrap.cc:492] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:30.016213 1064 ts_tablet_manager.cc:1403] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.001s
I20260810 07:53:30.029969 3702 raft_consensus.cc:493] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.030649 3702 raft_consensus.cc:515] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.033954 3702 leader_election.cc:290] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:30.035028 1047 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:30.036648 3776 leader_election.cc:290] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:30.040211 308 leader_election.cc:343] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:30.029856 1064 raft_consensus.cc:359] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.040706 308 leader_election.cc:304] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:30.041167 1064 raft_consensus.cc:385] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.041579 1064 raft_consensus.cc:740] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.042091 3755 raft_consensus.cc:2749] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:30.042258 1064 consensus_queue.cc:260] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.050343 1064 ts_tablet_manager.cc:1434] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.034s user 0.007s sys 0.001s
I20260810 07:53:30.078322 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.080958 3677 leader_election.cc:290] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.085561 1064 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:30.102314 3766 raft_consensus.cc:493] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.102917 3766 raft_consensus.cc:515] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.107916 1064 tablet_bootstrap.cc:654] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:30.121063 3766 leader_election.cc:290] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.124310 1047 tablet_bootstrap.cc:654] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:30.125083 309 leader_election.cc:343] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:30.124330 897 raft_consensus.cc:3060] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.163478 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.168970 1064 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:30.169521 1064 ts_tablet_manager.cc:1403] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.006s
I20260810 07:53:30.172691 1064 raft_consensus.cc:359] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.177673 1064 raft_consensus.cc:385] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.177989 1064 raft_consensus.cc:740] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.178715 1064 consensus_queue.cc:260] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.182197 1047 tablet_bootstrap.cc:492] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:30.182665 1047 ts_tablet_manager.cc:1403] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.148s user 0.011s sys 0.002s
I20260810 07:53:30.194370 892 raft_consensus.cc:3060] T 0d7084712ff04ac48a54171534e04d69 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.203693 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.204397 892 raft_consensus.cc:2468] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:30.208868 3776 raft_consensus.cc:493] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.209434 3776 raft_consensus.cc:515] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.193323 3702 raft_consensus.cc:493] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.221122 3702 raft_consensus.cc:515] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.223625 3702 leader_election.cc:290] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:30.225054 1064 ts_tablet_manager.cc:1434] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.055s user 0.004s sys 0.003s
W20260810 07:53:30.246558 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 96643.5us
I20260810 07:53:30.231685 1071 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:30.232749 308 leader_election.cc:343] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:30.280758 308 leader_election.cc:304] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:30.281616 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 48916.2us
I20260810 07:53:30.282838 3820 raft_consensus.cc:2749] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:30.297442 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:30.298171 411 raft_consensus.cc:2468] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:30.299118 1071 tablet_bootstrap.cc:654] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:30.335906 1047 raft_consensus.cc:359] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.336592 1047 raft_consensus.cc:385] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.336876 1047 raft_consensus.cc:740] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.337450 1047 consensus_queue.cc:260] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.416255 2982 mvcc.cc:204] Tried to move back new op lower bound from 7316883046402297856 to 7316883031338127360. Current Snapshot: MvccSnapshot[applied={T|T < 7316883042212466688 or (T in {7316883045803274240})}]
W20260810 07:53:30.430776 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 85063us
I20260810 07:53:30.444154 1071 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:30.450985 573 leader_election.cc:304] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:30.466145 1047 ts_tablet_manager.cc:1434] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.283s user 0.005s sys 0.011s
I20260810 07:53:30.484586 1064 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:30.498661 3776 leader_election.cc:290] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:30.522531 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:30.444639 1071 ts_tablet_manager.cc:1403] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.213s user 0.005s sys 0.008s
I20260810 07:53:30.467267 1047 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:30.537381 3515 raft_consensus.cc:2804] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:30.538466 3515 raft_consensus.cc:493] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260810 07:53:30.538601 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.654s (SENT)
I20260810 07:53:30.539063 3515 raft_consensus.cc:3060] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.539225 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:30.542001 1064 tablet_bootstrap.cc:654] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:30.548960 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 98051.1us
I20260810 07:53:30.561863 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.566300 3825 consensus_queue.cc:1048] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:30.606645 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.607697 3834 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:30.608675 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 70118.4us
I20260810 07:53:30.615175 3766 raft_consensus.cc:493] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.615897 3766 raft_consensus.cc:515] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.621719 3766 leader_election.cc:290] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.622581 1047 tablet_bootstrap.cc:654] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:30.626667 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:30.627501 411 raft_consensus.cc:2468] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:30.630774 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:30.632091 411 raft_consensus.cc:3060] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.628695 1071 raft_consensus.cc:359] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.644325 3790 consensus_queue.cc:1048] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:30.645936 3677 consensus_queue.cc:1048] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:30.648862 3660 raft_consensus.cc:493] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.649389 3660 raft_consensus.cc:515] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.654619 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.655287 670 raft_consensus.cc:2468] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:30.648044 3677 consensus_queue.cc:1048] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260810 07:53:30.658036 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:30.658687 671 raft_consensus.cc:3060] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.640698 411 raft_consensus.cc:2468] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:30.663524 3515 raft_consensus.cc:515] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.664830 3660 leader_election.cc:290] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.651631 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:30.669723 417 raft_consensus.cc:2468] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:30.640882 1071 raft_consensus.cc:385] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.645066 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:30.670754 409 raft_consensus.cc:3060] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.670486 1071 raft_consensus.cc:740] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.671692 1071 consensus_queue.cc:260] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.674408 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:30.675087 664 raft_consensus.cc:3060] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.690284 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.694594 3515 leader_election.cc:290] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:30.694869 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 51443.7us
I20260810 07:53:30.695071 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.695925 677 raft_consensus.cc:2468] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:30.712026 1064 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:30.712687 1064 ts_tablet_manager.cc:1403] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.257s user 0.012s sys 0.008s
I20260810 07:53:30.725675 671 raft_consensus.cc:2468] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:30.742913 1064 raft_consensus.cc:359] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.726732 664 raft_consensus.cc:2468] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:30.748093 1064 raft_consensus.cc:385] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:30.749631 1064 raft_consensus.cc:740] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:30.643450 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a91af48fe6a4bd986b5266f09fad6b2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:30.750519 3755 raft_consensus.cc:493] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.686007 409 raft_consensus.cc:2468] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
W20260810 07:53:30.705037 308 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:30.751178 3755 raft_consensus.cc:515] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:30.751819 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46844.9us
I20260810 07:53:30.753278 1064 consensus_queue.cc:260] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:30.755348 3755 leader_election.cc:290] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:30.759516 792 leader_election.cc:304] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:30.761811 1064 ts_tablet_manager.cc:1434] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.049s user 0.004s sys 0.004s
I20260810 07:53:30.766083 3830 raft_consensus.cc:2804] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:30.766578 3830 raft_consensus.cc:493] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:30.767073 3830 raft_consensus.cc:3060] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.772203 792 leader_election.cc:304] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:30.773963 1071 ts_tablet_manager.cc:1434] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.243s user 0.007s sys 0.000s
I20260810 07:53:30.789260 3831 raft_consensus.cc:2804] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:30.789772 3831 raft_consensus.cc:493] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:30.790171 3831 raft_consensus.cc:3060] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.805681 1064 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:30.825949 792 leader_election.cc:304] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:30.827121 3766 raft_consensus.cc:2804] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:30.828248 3766 raft_consensus.cc:697] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:30.829427 3766 consensus_queue.cc:237] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.833171 792 leader_election.cc:304] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:30.834125 3776 raft_consensus.cc:2804] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:30.834771 3776 raft_consensus.cc:697] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:30.835806 3776 consensus_queue.cc:237] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.838449 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:30.839272 410 raft_consensus.cc:2393] T 93f62494bb034cd0b4f34f61652569c6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
W20260810 07:53:30.839350 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.591s (SENT)
I20260810 07:53:30.839978 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:30.859206 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:30.859898 410 raft_consensus.cc:3060] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.860988 3515 consensus_queue.cc:1048] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:30.864523 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:30.782027 3830 raft_consensus.cc:515] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.869410 410 raft_consensus.cc:2468] T 6d67fef331994ca48f1dfce1e3b039f4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:30.869650 3831 raft_consensus.cc:515] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.877772 3845 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:30.879680 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69f1405e9526433a8fd727ed237f8c84" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:30.882809 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 43525us
I20260810 07:53:30.908390 3830 leader_election.cc:290] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.927323 1071 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:30.931167 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f62494bb034cd0b4f34f61652569c6" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.944697 3843 raft_consensus.cc:493] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:30.945314 3843 raft_consensus.cc:515] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:30.946386 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.948663 3843 leader_election.cc:290] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:30.872571 3831 leader_election.cc:290] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:30.956293 1064 tablet_bootstrap.cc:654] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:30.957190 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d67fef331994ca48f1dfce1e3b039f4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:30.957859 894 raft_consensus.cc:3060] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:30.967934 894 raft_consensus.cc:2468] T 6d67fef331994ca48f1dfce1e3b039f4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:30.979128 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:30.979728 891 raft_consensus.cc:2468] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:30.976459 575 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.126:38193, user_credentials={real_user=slave}} blocked reactor thread for 72527us
W20260810 07:53:31.023375 309 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.031445 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:31.060938 309 leader_election.cc:304] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:31.062216 3843 raft_consensus.cc:2749] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:31.070688 1064 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:31.070753 1071 tablet_bootstrap.cc:654] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.071215 1064 ts_tablet_manager.cc:1403] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.266s user 0.012s sys 0.002s
W20260810 07:53:31.085110 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 61770.8us
I20260810 07:53:31.081561 573 leader_election.cc:304] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:31.091958 792 leader_election.cc:304] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:31.093963 3830 raft_consensus.cc:2749] T 2a91af48fe6a4bd986b5266f09fad6b2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:31.097879 3515 raft_consensus.cc:2804] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:31.098335 3515 raft_consensus.cc:697] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:31.099076 3515 consensus_queue.cc:237] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:31.120573 1064 raft_consensus.cc:359] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.121438 1064 raft_consensus.cc:385] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.121757 1064 raft_consensus.cc:740] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.101207 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:31.144938 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 49084.6us
W20260810 07:53:31.145744 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:31.147455 574 leader_election.cc:304] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:31.150882 1064 consensus_queue.cc:260] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.153654 1064 ts_tablet_manager.cc:1434] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.082s user 0.008s sys 0.000s
I20260810 07:53:31.154704 1064 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:31.174213 3825 raft_consensus.cc:2749] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:31.184070 3830 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:31.184883 3830 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.187803 3830 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:31.188874 792 leader_election.cc:304] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:31.197461 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55aaf296555b43f9a27f2926054c8d4b consensus state has no leader
I20260810 07:53:31.197307 3830 raft_consensus.cc:2749] T 69f1405e9526433a8fd727ed237f8c84 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:31.202720 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 55371.9us
I20260810 07:53:31.211745 1047 tablet_bootstrap.cc:492] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:31.212635 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:31.213200 666 raft_consensus.cc:3060] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:31.220449 308 leader_election.cc:343] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:31.224304 308 leader_election.cc:343] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.226775 1064 tablet_bootstrap.cc:654] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.231022 666 raft_consensus.cc:2468] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:31.212244 1047 ts_tablet_manager.cc:1403] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.745s user 0.025s sys 0.018s
W20260810 07:53:31.235455 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 132954us
I20260810 07:53:31.246323 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:31.246968 664 raft_consensus.cc:3060] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:31.255229 664 raft_consensus.cc:2468] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
W20260810 07:53:31.267266 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 62964us
I20260810 07:53:31.270946 792 leader_election.cc:304] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:31.271777 3830 raft_consensus.cc:2804] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:31.272285 3830 raft_consensus.cc:697] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:31.273571 3830 consensus_queue.cc:237] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:31.267160 1047 raft_consensus.cc:359] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:31.270148 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:31.281745 1047 raft_consensus.cc:385] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.282068 1047 raft_consensus.cc:740] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.282652 1047 consensus_queue.cc:260] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:31.287567 1047 ts_tablet_manager.cc:1434] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.055s user 0.003s sys 0.003s
I20260810 07:53:31.288815 1047 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:31.303154 1047 tablet_bootstrap.cc:654] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.309011 1064 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:31.317058 1064 ts_tablet_manager.cc:1403] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.163s user 0.010s sys 0.006s
W20260810 07:53:31.320871 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 227815us
I20260810 07:53:31.322117 792 leader_election.cc:304] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:31.322945 3830 raft_consensus.cc:2804] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:31.323539 3830 raft_consensus.cc:697] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:31.324357 3830 consensus_queue.cc:237] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:31.329365 792 leader_election.cc:343] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.331682 3881 tablet.cc:1724] T 98dec9b072ac49959181003cb78567af P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:31.340257 1064 raft_consensus.cc:359] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.356871 1064 raft_consensus.cc:385] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.357225 1064 raft_consensus.cc:740] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.389219 1064 consensus_queue.cc:260] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.391402 1064 ts_tablet_manager.cc:1434] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.074s user 0.004s sys 0.003s
I20260810 07:53:31.422784 1047 tablet_bootstrap.cc:492] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:31.423349 1047 ts_tablet_manager.cc:1403] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.004s
I20260810 07:53:31.429915 1047 raft_consensus.cc:359] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.430987 1047 raft_consensus.cc:385] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.431339 1047 raft_consensus.cc:740] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.445995 3892 tablet.cc:1724] T 98dec9b072ac49959181003cb78567af P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:31.447006 1071 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:31.447463 1071 ts_tablet_manager.cc:1403] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.520s user 0.011s sys 0.002s
I20260810 07:53:31.464236 1071 raft_consensus.cc:359] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.464996 1071 raft_consensus.cc:385] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.465274 1071 raft_consensus.cc:740] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.465915 1071 consensus_queue.cc:260] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.432129 1047 consensus_queue.cc:260] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.469187 1064 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:31.470978 1047 ts_tablet_manager.cc:1434] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20260810 07:53:31.473888 3881 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 381) took 1064 ms. Trace:
W20260810 07:53:31.480011 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 136806us
I20260810 07:53:31.491958 1064 tablet_bootstrap.cc:654] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.474351 3881 rpcz_store.cc:276] 0810 07:53:30.409696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:30.410015 (+ 319us) service_pool.cc:224] Handling call
0810 07:53:31.473817 (+1063802us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:30.412053 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:30.412589 (+ 536us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:30.412646 (+ 57us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:30.412708 (+ 62us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:30.412869 (+ 161us) alter_schema_op.cc:128] START. Timestamp: 1786348410412769
0810 07:53:30.412905 (+ 36us) op_driver.cc:348] REPLICATION: starting
0810 07:53:30.413584 (+ 679us) log.cc:844] Serialized 173 byte log entry
0810 07:53:31.271277 (+857693us) op_driver.cc:464] REPLICATION: finished
0810 07:53:31.331578 (+ 60301us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:31.347362 (+ 15784us) log.cc:844] Serialized 5 byte log entry
0810 07:53:31.443732 (+ 96370us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:31.444612 (+ 880us) log.cc:844] Serialized 14 byte log entry
0810 07:53:31.473644 (+ 29032us) alter_schema_op.cc:73] Released schema lock
0810 07:53:31.473692 (+ 48us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":60084,"prepare.queue_time_us":245,"prepare.run_cpu_time_us":8648,"prepare.run_wall_time_us":65014,"replication_time_us":858234,"thread_start_us":125024,"threads_started":2,"wal-append.queue_time_us":61812}]]}
I20260810 07:53:31.505316 1071 ts_tablet_manager.cc:1434] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.058s user 0.008s sys 0.000s
I20260810 07:53:31.529413 1071 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:31.544286 1047 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:31.559370 3897 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
W20260810 07:53:31.569851 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 248217us
I20260810 07:53:31.583262 1047 tablet_bootstrap.cc:654] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.585700 3896 tablet.cc:1724] T 98dec9b072ac49959181003cb78567af P 60781ef3c8b84a8788f43d77e1bb3b19: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:31.594468 1064 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:31.594957 1064 ts_tablet_manager.cc:1403] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.126s user 0.004s sys 0.008s
I20260810 07:53:31.596165 3820 raft_consensus.cc:493] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:31.596696 3820 raft_consensus.cc:515] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.597029 3897 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:31.599963 3820 leader_election.cc:290] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:31.621783 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:31.622728 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:31.630357 1064 raft_consensus.cc:359] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.631273 1064 raft_consensus.cc:385] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.631625 1064 raft_consensus.cc:740] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.635222 3897 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
W20260810 07:53:31.641001 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 46897.3us
I20260810 07:53:31.646950 1064 consensus_queue.cc:260] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.652086 1071 tablet_bootstrap.cc:654] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:31.654364 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260810 07:53:31.689026 308 leader_election.cc:343] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:31.684141 309 leader_election.cc:343] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.691926 309 leader_election.cc:304] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:31.693452 3887 raft_consensus.cc:2749] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:31.696269 1064 ts_tablet_manager.cc:1434] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.101s user 0.008s sys 0.000s
I20260810 07:53:31.705603 1064 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:31.710428 309 leader_election.cc:304] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:31.712100 3887 raft_consensus.cc:2804] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:31.712601 3887 raft_consensus.cc:493] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:31.713106 3887 raft_consensus.cc:3060] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:31.720845 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 85247.3us
I20260810 07:53:31.748248 1064 tablet_bootstrap.cc:654] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:31.763882 3788 raft_consensus.cc:493] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:31.764405 3788 raft_consensus.cc:515] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:31.773290 3788 leader_election.cc:290] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:31.788513 3766 raft_consensus.cc:493] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:31.789126 3766 raft_consensus.cc:515] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.791164 3766 leader_election.cc:290] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:31.793579 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 44219.9us
I20260810 07:53:31.803131 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:31.817039 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 105573us
I20260810 07:53:31.819413 1064 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:31.819999 1064 ts_tablet_manager.cc:1403] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.002s
I20260810 07:53:31.823354 1064 raft_consensus.cc:359] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.825893 1064 raft_consensus.cc:385] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.849324 1071 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:31.849851 1071 ts_tablet_manager.cc:1403] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.321s user 0.003s sys 0.010s
I20260810 07:53:31.828344 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:31.838186 792 leader_election.cc:343] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.850597 1064 raft_consensus.cc:740] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.851799 1064 consensus_queue.cc:260] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.852869 877 raft_consensus.cc:2468] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:31.866851 1064 ts_tablet_manager.cc:1434] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20260810 07:53:31.867934 1064 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:31.877018 3830 raft_consensus.cc:493] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:31.877691 3830 raft_consensus.cc:515] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.881466 3830 leader_election.cc:290] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:31.911664 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:31.913570 3830 consensus_queue.cc:1048] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:31.937713 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 198756us
W20260810 07:53:31.939978 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 196923us
I20260810 07:53:31.942067 3766 consensus_queue.cc:1048] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:31.950203 1047 tablet_bootstrap.cc:492] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:31.950692 1047 ts_tablet_manager.cc:1403] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.455s user 0.007s sys 0.005s
I20260810 07:53:31.962984 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b705e1630f124370af386436f325004b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:31.963608 409 raft_consensus.cc:3060] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:31.966225 1071 raft_consensus.cc:359] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.967372 1071 raft_consensus.cc:385] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.967728 1071 raft_consensus.cc:740] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:31.948622 1064 tablet_bootstrap.cc:654] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:31.976830 792 leader_election.cc:343] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:31.991132 1047 raft_consensus.cc:359] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:31.991798 1047 raft_consensus.cc:385] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:31.992062 1047 raft_consensus.cc:740] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:31.993803 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 55406.6us
W20260810 07:53:31.995043 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 54453.7us
I20260810 07:53:32.008328 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.009094 407 raft_consensus.cc:2468] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:31.930384 3887 raft_consensus.cc:515] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:32.015779 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:32.016484 414 raft_consensus.cc:3060] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.031503 3909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:32.032020 3909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:32.034468 409 raft_consensus.cc:2468] T b705e1630f124370af386436f325004b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:31.992622 1047 consensus_queue.cc:260] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.061911 1064 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:32.062426 1064 ts_tablet_manager.cc:1403] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.195s user 0.008s sys 0.003s
I20260810 07:53:32.067873 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.067631 1064 raft_consensus.cc:359] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.068359 1064 raft_consensus.cc:385] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.068774 1064 raft_consensus.cc:740] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.070602 411 raft_consensus.cc:2393] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:32.077313 3887 leader_election.cc:290] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.079872 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:32.086617 1071 consensus_queue.cc:260] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.088447 1071 ts_tablet_manager.cc:1434] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.238s user 0.004s sys 0.002s
I20260810 07:53:32.097060 1047 ts_tablet_manager.cc:1434] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.146s user 0.006s sys 0.001s
I20260810 07:53:32.098074 1047 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:32.106516 1071 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:32.097087 414 raft_consensus.cc:2468] T b2fb91a223154dde8ecc2bebf2c0fff9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:32.112892 3830 consensus_queue.cc:1048] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:32.116559 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.117321 411 raft_consensus.cc:2468] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.135967 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.137758 414 raft_consensus.cc:2468] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.069525 1064 consensus_queue.cc:260] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.139178 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:32.139750 677 raft_consensus.cc:3060] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.139997 1064 ts_tablet_manager.cc:1434] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.077s user 0.007s sys 0.000s
I20260810 07:53:32.134090 3909 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:32.182478 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.183275 877 raft_consensus.cc:2468] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
W20260810 07:53:32.194634 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:32.216511 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:32.217316 886 raft_consensus.cc:3060] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.218467 1064 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:32.231585 886 raft_consensus.cc:2468] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:32.236248 1071 tablet_bootstrap.cc:654] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.241386 677 raft_consensus.cc:2468] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:32.257088 1047 tablet_bootstrap.cc:654] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.265399 308 leader_election.cc:304] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:32.266425 3820 raft_consensus.cc:2804] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:32.266996 3820 raft_consensus.cc:697] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:32.268003 3820 consensus_queue.cc:237] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:32.272132 1064 tablet_bootstrap.cc:654] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:32.302122 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 66752.5us
I20260810 07:53:32.303896 792 leader_election.cc:304] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:32.305374 3916 raft_consensus.cc:2804] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:32.305778 3916 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:32.306124 3916 raft_consensus.cc:3060] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:32.316874 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 61988.9us
I20260810 07:53:32.328042 3811 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.328948 3916 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.328827 3811 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.331775 3811 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.337137 3916 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:32.337952 2560 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.717s (SENT) [suppressed 1 similar messages]
I20260810 07:53:32.359334 792 leader_election.cc:304] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:32.360920 3916 raft_consensus.cc:2804] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:32.361761 3916 raft_consensus.cc:493] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:32.362115 3830 consensus_queue.cc:1048] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:32.362741 3811 raft_consensus.cc:493] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.363279 3811 raft_consensus.cc:515] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.360582 3919 consensus_queue.cc:1048] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260810 07:53:32.362262 3916 raft_consensus.cc:3060] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.388520 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:32.389631 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:32.400393 1064 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:32.401162 1064 ts_tablet_manager.cc:1403] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.183s user 0.017s sys 0.002s
I20260810 07:53:32.402064 3926 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.402663 3926 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.405696 3824 raft_consensus.cc:493] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.406399 3926 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.406308 3824 raft_consensus.cc:515] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.406710 1064 raft_consensus.cc:359] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.407452 1064 raft_consensus.cc:385] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.407727 1064 raft_consensus.cc:740] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.408550 1064 consensus_queue.cc:260] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.409494 3824 leader_election.cc:290] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.411341 3811 leader_election.cc:290] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.417130 1064 ts_tablet_manager.cc:1434] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.016s user 0.007s sys 0.001s
I20260810 07:53:32.418073 1064 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:32.444257 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:32.445144 671 raft_consensus.cc:3060] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.447829 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.448500 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.448637 407 raft_consensus.cc:2468] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:32.449209 409 raft_consensus.cc:2468] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:32.452351 1047 tablet_bootstrap.cc:492] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:32.452931 1047 ts_tablet_manager.cc:1403] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.355s user 0.010s sys 0.006s
I20260810 07:53:32.455576 1047 raft_consensus.cc:359] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.456271 1047 raft_consensus.cc:385] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.466650 1064 tablet_bootstrap.cc:654] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.474097 309 leader_election.cc:304] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:32.475054 3811 raft_consensus.cc:2804] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:32.474573 3916 raft_consensus.cc:515] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.475574 3811 raft_consensus.cc:493] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:32.475966 3811 raft_consensus.cc:3060] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.476145 671 raft_consensus.cc:2468] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:32.483852 1064 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:32.484594 1064 ts_tablet_manager.cc:1403] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.006s
I20260810 07:53:32.489377 1071 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:32.489864 1071 ts_tablet_manager.cc:1403] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.384s user 0.011s sys 0.001s
I20260810 07:53:32.456559 1047 raft_consensus.cc:740] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.490942 1047 consensus_queue.cc:260] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.493530 1047 ts_tablet_manager.cc:1434] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260810 07:53:32.494607 1047 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:32.457228 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 119305us
W20260810 07:53:32.499677 308 leader_election.cc:343] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:32.500643 308 leader_election.cc:343] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:32.525597 3925 consensus_queue.cc:1048] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:32.527048 1064 raft_consensus.cc:359] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.527650 1064 raft_consensus.cc:385] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.527448 574 leader_election.cc:304] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19
I20260810 07:53:32.527886 1064 raft_consensus.cc:740] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.528955 3925 raft_consensus.cc:2764] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 07:53:32.529165 3920 raft_consensus.cc:493] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.528461 1064 consensus_queue.cc:260] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.529641 3920 raft_consensus.cc:515] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.532097 3920 leader_election.cc:290] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.533283 1064 ts_tablet_manager.cc:1434] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260810 07:53:32.534529 1064 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:32.535674 3931 consensus_queue.cc:1048] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:32.542582 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 61209.6us
I20260810 07:53:32.542760 1064 tablet_bootstrap.cc:654] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.553750 792 leader_election.cc:304] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:32.554543 3931 raft_consensus.cc:2804] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:32.555011 3931 raft_consensus.cc:697] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
W20260810 07:53:32.556394 576 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 47407.2us
I20260810 07:53:32.555894 3931 consensus_queue.cc:237] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.567135 3916 leader_election.cc:290] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.579411 573 leader_election.cc:304] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:32.492378 1071 raft_consensus.cc:359] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.590528 1071 raft_consensus.cc:385] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.590881 1071 raft_consensus.cc:740] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.594648 1047 tablet_bootstrap.cc:654] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.597010 3927 raft_consensus.cc:2764] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 07:53:32.600502 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:32.603533 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:32.604041 677 raft_consensus.cc:3060] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.609107 573 leader_election.cc:304] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:32.622787 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260810 07:53:32.630470 792 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 47372us
W20260810 07:53:32.633234 792 leader_election.cc:343] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:32.650110 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 79528.3us
I20260810 07:53:32.661064 1071 consensus_queue.cc:260] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.686658 3811 raft_consensus.cc:515] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.694795 3931 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.695670 1064 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:32.695351 3931 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.696362 1064 ts_tablet_manager.cc:1403] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.162s user 0.017s sys 0.011s
I20260810 07:53:32.698205 3931 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.701324 1064 raft_consensus.cc:359] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.702023 1064 raft_consensus.cc:385] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.702416 1064 raft_consensus.cc:740] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.703423 1064 consensus_queue.cc:260] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:32.705232 3950 log.cc:927] Time spent T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d: Append to log took a long time: real 0.065s user 0.001s sys 0.000s
I20260810 07:53:32.710572 1064 ts_tablet_manager.cc:1434] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260810 07:53:32.715734 1064 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:32.716550 1047 tablet_bootstrap.cc:492] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:32.717060 1047 ts_tablet_manager.cc:1403] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.223s user 0.012s sys 0.002s
I20260810 07:53:32.724912 3931 raft_consensus.cc:493] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.725548 3931 raft_consensus.cc:515] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.728772 3931 leader_election.cc:290] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.734479 1071 ts_tablet_manager.cc:1434] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.244s user 0.007s sys 0.003s
W20260810 07:53:32.735087 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 63183.9us
I20260810 07:53:32.754652 1064 tablet_bootstrap.cc:654] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.744611 3811 leader_election.cc:290] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:32.757742 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 44110.6us
W20260810 07:53:32.778134 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 127382us
W20260810 07:53:32.806615 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 43792.7us
I20260810 07:53:32.811298 792 leader_election.cc:304] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:32.812536 3931 raft_consensus.cc:2804] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:32.812942 3931 raft_consensus.cc:493] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:32.813318 3931 raft_consensus.cc:3060] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.820222 3954 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.820752 3954 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.823000 3954 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.824360 1071 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:32.830826 1064 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:32.831276 1064 ts_tablet_manager.cc:1403] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.116s user 0.000s sys 0.011s
I20260810 07:53:32.831105 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.831845 886 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:32.843436 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:32.844003 895 raft_consensus.cc:3060] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.844702 1064 raft_consensus.cc:359] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.845429 1064 raft_consensus.cc:385] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:32.845682 1064 raft_consensus.cc:740] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:32.846375 1064 consensus_queue.cc:260] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.852690 895 raft_consensus.cc:2468] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:32.856395 3916 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.853025 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.862491 3916 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.862965 886 raft_consensus.cc:2468] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:32.864351 3916 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.864765 3931 raft_consensus.cc:515] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.867571 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:32.868144 409 raft_consensus.cc:3060] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.870362 1071 tablet_bootstrap.cc:654] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.876348 409 raft_consensus.cc:2468] T aed14fa26b9a46a4b3e9a43bac05a795 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
W20260810 07:53:32.893085 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 268296us
I20260810 07:53:32.900687 3931 leader_election.cc:290] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:32.878068 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.901654 411 raft_consensus.cc:2468] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.879582 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:32.881047 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.902627 415 raft_consensus.cc:3060] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.902681 413 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.881173 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:32.902932 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 293769us
I20260810 07:53:32.903462 417 raft_consensus.cc:2468] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.896793 3958 raft_consensus.cc:493] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:32.909215 3958 raft_consensus.cc:515] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:32.910835 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.912308 3958 leader_election.cc:290] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:32.913545 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.914279 877 raft_consensus.cc:2393] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:32.916775 415 raft_consensus.cc:2468] T f65480f06a3c4b2886aa2485f81e16f2 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:32.878059 1064 ts_tablet_manager.cc:1434] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.046s user 0.007s sys 0.000s
I20260810 07:53:32.929148 1064 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:32.952082 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:32.952821 410 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:32.954231 410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:32.954757 410 raft_consensus.cc:3060] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.963056 410 raft_consensus.cc:2468] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:32.969282 3916 consensus_queue.cc:1048] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:32.971472 3916 consensus_queue.cc:1048] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:32.976945 792 leader_election.cc:304] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:32.977762 3916 raft_consensus.cc:2804] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:32.978210 3916 raft_consensus.cc:493] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:32.978657 3916 raft_consensus.cc:3060] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:32.985776 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.986527 886 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:32.991081 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:32.991753 895 raft_consensus.cc:2468] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:32.994807 1064 tablet_bootstrap.cc:654] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:32.994820 792 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.000025 792 leader_election.cc:304] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.000761 3931 raft_consensus.cc:2804] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.001252 3931 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.001669 3931 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.005050 3920 raft_consensus.cc:2804] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.005388 3920 raft_consensus.cc:493] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.005699 3920 raft_consensus.cc:3060] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.010497 1071 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:33.010931 1071 ts_tablet_manager.cc:1403] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.206s user 0.003s sys 0.007s
I20260810 07:53:33.013206 3920 raft_consensus.cc:515] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.015269 3920 leader_election.cc:290] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:33.016032 792 leader_election.cc:304] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.017455 3934 raft_consensus.cc:2804] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.017545 792 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.017957 3934 raft_consensus.cc:697] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:33.019150 3920 raft_consensus.cc:2804] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.018828 3934 consensus_queue.cc:237] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.019531 3920 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.019860 3920 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.031457 3916 raft_consensus.cc:515] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.045372 1047 raft_consensus.cc:359] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.046452 1047 raft_consensus.cc:385] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.046754 1047 raft_consensus.cc:740] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.047490 1047 consensus_queue.cc:260] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:33.061343 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 140646us
I20260810 07:53:33.066032 1047 ts_tablet_manager.cc:1434] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.348s user 0.001s sys 0.007s
I20260810 07:53:33.068024 1047 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:33.060434 677 raft_consensus.cc:2468] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:33.070704 3677 raft_consensus.cc:2764] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260810 07:53:33.073696 32455 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:33.088050 3825 consensus_queue.cc:1048] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:33.109157 1071 raft_consensus.cc:359] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.109819 1071 raft_consensus.cc:385] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.110086 1071 raft_consensus.cc:740] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.122821 1047 tablet_bootstrap.cc:654] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.116545 309 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.123967 3958 raft_consensus.cc:2804] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.124383 3958 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.124795 3958 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.125046 1071 consensus_queue.cc:260] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.132301 1071 ts_tablet_manager.cc:1434] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.121s user 0.007s sys 0.000s
I20260810 07:53:33.133324 1071 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:33.140105 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:33.102516 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 85024.4us
I20260810 07:53:33.108266 3920 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.142454 3920 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:33.142344 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.143466 792 leader_election.cc:304] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.143822 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.144146 3934 raft_consensus.cc:2804] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.144883 3934 raft_consensus.cc:697] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:33.147262 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.145879 3934 consensus_queue.cc:237] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.153708 3916 leader_election.cc:290] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:33.155740 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.157049 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.157630 666 raft_consensus.cc:3060] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.168520 309 leader_election.cc:304] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.169361 3955 raft_consensus.cc:2804] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.170120 3955 raft_consensus.cc:697] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:33.171394 3955 consensus_queue.cc:237] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.176497 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.141491 3931 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.182576 3931 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:33.183311 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.187485 1064 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:33.187949 1064 ts_tablet_manager.cc:1403] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.259s user 0.018s sys 0.008s
I20260810 07:53:33.192962 666 raft_consensus.cc:2468] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:33.192104 1064 raft_consensus.cc:359] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.193471 1064 raft_consensus.cc:385] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.193738 1064 raft_consensus.cc:740] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.193710 3958 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.194417 1064 consensus_queue.cc:260] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.195806 1064 ts_tablet_manager.cc:1434] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260810 07:53:33.204843 309 leader_election.cc:304] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.206133 3954 raft_consensus.cc:2804] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.206530 3954 raft_consensus.cc:493] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.206872 3954 raft_consensus.cc:3060] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.214593 3954 raft_consensus.cc:515] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.217537 3954 leader_election.cc:290] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.219017 309 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.220252 3820 raft_consensus.cc:2804] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.220636 3820 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.221007 3820 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.233187 3820 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.196519 3958 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.245632 309 leader_election.cc:304] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
W20260810 07:53:33.246759 309 consensus_peers.cc:597] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 371fa2534984499b95c0945af358e8fc. This is attempt 6: this message will repeat every 5th retry.
I20260810 07:53:33.246836 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.247112 3912 raft_consensus.cc:2804] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:33.247447 3912 raft_consensus.cc:493] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:33.247749 3912 raft_consensus.cc:3060] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.250168 3820 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.259063 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:33.261902 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.717s (SENT)
I20260810 07:53:33.262336 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:33.263234 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.266486 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:33.274149 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.537s (SENT)
I20260810 07:53:33.274566 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
W20260810 07:53:33.276678 308 leader_election.cc:343] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.286428 308 leader_election.cc:343] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.288686 1047 tablet_bootstrap.cc:492] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:33.289214 1047 ts_tablet_manager.cc:1403] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.222s user 0.012s sys 0.001s
W20260810 07:53:33.293365 308 leader_election.cc:343] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.196703 1064 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:33.316839 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:33.317433 885 raft_consensus.cc:3060] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.317876 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:33.256328 3912 raft_consensus.cc:515] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.318589 897 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:33.319893 3912 leader_election.cc:290] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.321382 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:33.322218 888 raft_consensus.cc:2393] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60781ef3c8b84a8788f43d77e1bb3b19 in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:33.325222 885 raft_consensus.cc:2468] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:33.333076 1047 raft_consensus.cc:359] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.333345 3986 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:33.333760 1047 raft_consensus.cc:385] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.334036 1047 raft_consensus.cc:740] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.334638 1047 consensus_queue.cc:260] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:33.334944 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 73096.9us
I20260810 07:53:33.336160 1047 ts_tablet_manager.cc:1434] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.047s user 0.002s sys 0.000s
I20260810 07:53:33.343672 1071 tablet_bootstrap.cc:654] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.344813 3988 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:33.347930 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.348624 677 raft_consensus.cc:3060] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.351722 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:33.358057 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 83950.8us
I20260810 07:53:33.356438 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:33.358891 407 raft_consensus.cc:3060] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.358256 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:33.365304 417 raft_consensus.cc:2393] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:33.370200 407 raft_consensus.cc:2468] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
W20260810 07:53:33.371855 308 leader_election.cc:343] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.372807 308 leader_election.cc:343] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.375303 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:33.388284 1047 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:33.390436 1071 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:33.390988 1071 ts_tablet_manager.cc:1403] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.258s user 0.007s sys 0.004s
W20260810 07:53:33.393158 308 leader_election.cc:343] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.407881 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:33.408771 885 raft_consensus.cc:3060] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:33.416085 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 213fa4838fe847de87091cfba97eaedc (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 213fa4838fe847de87091cfba97eaedc consensus state has no leader
I20260810 07:53:20.481794 2741 rpcz_store.cc:276] 0810 07:53:14.075930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:14.076127 (+ 197us) service_pool.cc:224] Handling call
0810 07:53:15.249577 (+1173450us) raft_consensus.cc:1471] Updating replica for 389 ops
0810 07:53:15.493760 (+244183us) raft_consensus.cc:1534] Early marking committed up to index 6
0810 07:53:15.533319 (+ 39559us) raft_consensus.cc:1539] Triggering prepare for 388 ops
0810 07:53:15.562854 (+ 29535us) write_op.cc:270] Start()
0810 07:53:15.563002 (+ 148us) write_op.cc:276] Timestamp: P: 1786348364873934 usec, L: 0
0810 07:53:15.608243 (+ 45241us) write_op.cc:270] Start()
0810 07:53:15.608360 (+ 117us) write_op.cc:276] Timestamp: P: 1786348364876419 usec, L: 0
0810 07:53:15.610092 (+ 1732us) write_op.cc:270] Start()
0810 07:53:15.610215 (+ 123us) write_op.cc:276] Timestamp: P: 1786348364891362 usec, L: 0
0810 07:53:15.610764 (+ 549us) write_op.cc:270] Start()
0810 07:53:15.610842 (+ 78us) write_op.cc:276] Timestamp: P: 1786348364952098 usec, L: 0
0810 07:53:15.611325 (+ 483us) write_op.cc:270] Start()
0810 07:53:15.611409 (+ 84us) write_op.cc:276] Timestamp: P: 1786348364975480 usec, L: 0
0810 07:53:15.611867 (+ 458us) write_op.cc:270] Start()
0810 07:53:15.611961 (+ 94us) write_op.cc:276] Timestamp: P: 1786348364979043 usec, L: 0
0810 07:53:15.612435 (+ 474us) write_op.cc:270] Start()
0810 07:53:15.612592 (+ 157us) write_op.cc:276] Timestamp: P: 1786348365114327 usec, L: 0
0810 07:53:15.665448 (+ 52856us) write_op.cc:270] Start()
0810 07:53:15.665573 (+ 125us) write_op.cc:276] Timestamp: P: 1786348365150143 usec, L: 0
0810 07:53:15.666195 (+ 622us) write_op.cc:270] Start()
0810 07:53:15.666294 (+ 99us) write_op.cc:276] Timestamp: P: 1786348365173643 usec, L: 0
0810 07:53:15.682124 (+ 15830us) write_op.cc:270] Start()
0810 07:53:15.682237 (+ 113us) write_op.cc:276] Timestamp: P: 1786348365177283 usec, L: 0
0810 07:53:15.682782 (+ 545us) write_op.cc:270] Start()
0810 07:53:15.682868 (+ 86us) write_op.cc:276] Timestamp: P: 1786348365214965 usec, L: 0
0810 07:53:15.683387 (+ 519us) write_op.cc:270] Start()
0810 07:53:15.683519 (+ 132us) write_op.cc:276] Timestamp: P: 1786348365237844 usec, L: 0
0810 07:53:15.684123 (+ 604us) write_op.cc:270] Start()
0810 07:53:15.684210 (+ 87us) write_op.cc:276] Timestamp: P: 1786348365249801 usec, L: 0
0810 07:53:15.715428 (+ 31218us) write_op.cc:270] Start()
0810 07:53:15.715566 (+ 138us) write_op.cc:276] Timestamp: P: 1786348365281841 usec, L: 0
0810 07:53:15.716532 (+ 966us) write_op.cc:270] Start()
0810 07:53:15.716628 (+ 96us) write_op.cc:276] Timestamp: P: 1786348365359884 usec, L: 0
0810 07:53:15.717210 (+ 582us) write_op.cc:270] Start()
0810 07:53:15.717287 (+ 77us) write_op.cc:276] Timestamp: P: 1786348365382748 usec, L: 0
0810 07:53:15.717768 (+ 481us) write_op.cc:270] Start()
0810 07:53:15.717861 (+ 93us) write_op.cc:276] Timestamp: P: 1786348365446104 usec, L: 0
0810 07:53:15.718304 (+ 443us) write_op.cc:270] Start()
0810 07:53:15.718379 (+ 75us) write_op.cc:276] Timestamp: P: 1786348365473310 usec, L: 0
0810 07:53:15.718788 (+ 409us) write_op.cc:270] Start()
0810 07:53:15.718870 (+ 82us) write_op.cc:276] Timestamp: P: 1786348365515396 usec, L: 0
0810 07:53:15.719293 (+ 423us) write_op.cc:270] Start()
0810 07:53:15.719363 (+ 70us) write_op.cc:276] Timestamp: P: 1786348365532341 usec, L: 0
0810 07:53:15.719759 (+ 396us) write_op.cc:270] Start()
0810 07:53:15.719836 (+ 77us) write_op.cc:276] Timestamp: P: 1786348365556650 usec, L: 0
0810 07:53:15.756981 (+ 37145us) write_op.cc:270] Start()
0810 07:53:15.757117 (+ 136us) write_op.cc:276] Timestamp: P: 1786348365591713 usec, L: 0
0810 07:53:15.757754 (+ 637us) write_op.cc:270] Start()
0810 07:53:15.757848 (+ 94us) write_op.cc:276] Timestamp: P: 1786348365594591 usec, L: 0
0810 07:53:15.758380 (+ 532us) write_op.cc:270] Start()
0810 07:53:15.758499 (+ 119us) write_op.cc:276] Timestamp: P: 1786348365698559 usec, L: 0
0810 07:53:15.759131 (+ 632us) write_op.cc:270] Start()
0810 07:53:15.759212 (+ 81us) write_op.cc:276] Timestamp: P: 1786348365734967 usec, L: 0
0810 07:53:15.759679 (+ 467us) write_op.cc:270] Start()
0810 07:53:15.759773 (+ 94us) write_op.cc:276] Timestamp: P: 1786348365779378 usec, L: 0
0810 07:53:15.760221 (+ 448us) write_op.cc:270] Start()
0810 07:53:15.760311 (+ 90us) write_op.cc:276] Timestamp: P: 1786348365835714 usec, L: 0
0810 07:53:15.792966 (+ 32655us) write_op.cc:270] Start()
0810 07:53:15.793081 (+ 115us) write_op.cc:276] Timestamp: P: 1786348365868495 usec, L: 0
0810 07:53:15.793782 (+ 701us) write_op.cc:270] Start()
0810 07:53:15.793868 (+ 86us) write_op.cc:276] Timestamp: P: 1786348365950207 usec, L: 0
0810 07:53:15.794364 (+ 496us) write_op.cc:270] Start()
0810 07:53:15.794446 (+ 82us) write_op.cc:276] Timestamp: P: 1786348365968886 usec, L: 0
0810 07:53:15.794916 (+ 470us) write_op.cc:270] Start()
0810 07:53:15.794991 (+ 75us) write_op.cc:276] Timestamp: P: 1786348365986448 usec, L: 0
0810 07:53:15.795478 (+ 487us) write_op.cc:270] Start()
0810 07:53:15.795555 (+ 77us) write_op.cc:276] Timestamp: P: 1786348366024040 usec, L: 0
0810 07:53:15.796255 (+ 700us) write_op.cc:270] Start()
0810 07:53:15.796333 (+ 78us) write_op.cc:276] Timestamp: P: 1786348366111941 usec, L: 0
0810 07:53:15.829131 (+ 32798us) write_op.cc:270] Start()
0810 07:53:15.829235 (+ 104us) write_op.cc:276] Timestamp: P: 1786348366132285 usec, L: 0
0810 07:53:15.829858 (+ 623us) write_op.cc:270] Start()
0810 07:53:15.829944 (+ 86us) write_op.cc:276] Timestamp: P: 1786348366147322 usec, L: 0
0810 07:53:15.830460 (+ 516us) write_op.cc:270] Start()
0810 07:53:15.830551 (+ 91us) write_op.cc:276] Timestamp: P: 1786348366160833 usec, L: 0
0810 07:53:15.831054 (+ 503us) write_op.cc:270] Start()
0810 07:53:15.831133 (+ 79us) write_op.cc:276] Timestamp: P: 1786348366173572 usec, L: 0
0810 07:53:15.831601 (+ 468us) write_op.cc:270] Start()
0810 07:53:15.831674 (+ 73us) write_op.cc:276] Timestamp: P: 1786348366203232 usec, L: 0
0810 07:53:15.849800 (+ 18126us) write_op.cc:270] Start()
0810 07:53:15.849901 (+ 101us) write_op.cc:276] Timestamp: P: 1786348366276918 usec, L: 0
0810 07:53:15.875996 (+ 26095us) write_op.cc:270] Start()
0810 07:53:15.876140 (+ 144us) write_op.cc:276] Timestamp: P: 1786348366281103 usec, L: 0
0810 07:53:15.876841 (+ 701us) write_op.cc:270] Start()
0810 07:53:15.876925 (+ 84us) write_op.cc:276] Timestamp: P: 1786348366288311 usec, L: 0
0810 07:53:15.877419 (+ 494us) write_op.cc:270] Start()
0810 07:53:15.877522 (+ 103us) write_op.cc:276] Timestamp: P: 1786348366341948 usec, L: 0
0810 07:53:15.878038 (+ 516us) write_op.cc:270] Start()
0810 07:53:15.878140 (+ 102us) write_op.cc:276] Timestamp: P: 1786348366370568 usec, L: 0
0810 07:53:15.889344 (+ 11204us) write_op.cc:270] Start()
0810 07:53:15.889458 (+ 114us) write_op.cc:276] Timestamp: P: 1786348366379882 usec, L: 0
0810 07:53:15.890043 (+ 585us) write_op.cc:270] Start()
0810 07:53:15.890135 (+ 92us) write_op.cc:276] Timestamp: P: 1786348366421065 usec, L: 0
0810 07:53:15.890627 (+ 492us) write_op.cc:270] Start()
0810 07:53:15.890707 (+ 80us) write_op.cc:276] Timestamp: P: 1786348366476908 usec, L: 0
0810 07:53:15.891201 (+ 494us) write_op.cc:270] Start()
0810 07:53:15.891295 (+ 94us) write_op.cc:276] Timestamp: P: 1786348366567990 usec, L: 0
0810 07:53:15.922733 (+ 31438us) write_op.cc:270] Start()
0810 07:53:15.922976 (+ 243us) write_op.cc:276] Timestamp: P: 1786348366617717 usec, L: 0
0810 07:53:15.923835 (+ 859us) write_op.cc:270] Start()
0810 07:53:15.923937 (+ 102us) write_op.cc:276] Timestamp: P: 1786348366657126 usec, L: 0
0810 07:53:15.924638 (+ 701us) write_op.cc:270] Start()
0810 07:53:16.011516 (+ 86878us) write_op.cc:276] Timestamp: P: 1786348366706036 usec, L: 0
0810 07:53:16.021545 (+ 10029us) write_op.cc:270] Start()
0810 07:53:16.021720 (+ 175us) write_op.cc:276] Timestamp: P: 1786348366722176 usec, L: 0
0810 07:53:16.022615 (+ 895us) write_op.cc:270] Start()
0810 07:53:16.022721 (+ 106us) write_op.cc:276] Timestamp: P: 1786348366730856 usec, L: 0
0810 07:53:16.023500 (+ 779us) write_op.cc:270] Start()
0810 07:53:16.023629 (+ 129us) write_op.cc:276] Timestamp: P: 1786348366762477 usec, L: 0
0810 07:53:16.160392 (+136763us) write_op.cc:270] Start()
0810 07:53:16.160781 (+ 389us) write_op.cc:276] Timestamp: P: 1786348366877062 usec, L: 0
0810 07:53:16.161512 (+ 731us) write_op.cc:270] Start()
0810 07:53:16.161618 (+ 106us) write_op.cc:276] Timestamp: P: 1786348366882548 usec, L: 0
0810 07:53:16.162217 (+ 599us) write_op.cc:270] Start()
0810 07:53:16.162314 (+ 97us) write_op.cc:276] Timestamp: P: 1786348367035710 usec, L: 0
0810 07:53:16.162899 (+ 585us) write_op.cc:270] Start()
0810 07:53:16.163004 (+ 105us) write_op.cc:276] Timestamp: P: 1786348367083196 usec, L: 0
0810 07:53:16.163615 (+ 611us) write_op.cc:270] Start()
0810 07:53:16.163719 (+ 104us) write_op.cc:276] Timestamp: P: 1786348367140457 usec, L: 0
0810 07:53:16.164366 (+ 647us) write_op.cc:270] Start()
0810 07:53:16.164474 (+ 108us) write_op.cc:276] Timestamp: P: 1786348367181809 usec, L: 0
0810 07:53:16.296871 (+132397us) write_op.cc:270] Start()
0810 07:53:16.297047 (+ 176us) write_op.cc:276] Timestamp: P: 1786348367228430 usec, L: 0
0810 07:53:16.297618 (+ 571us) write_op.cc:270] Start()
0810 07:53:16.297701 (+ 83us) write_op.cc:276] Timestamp: P: 1786348367265664 usec, L: 0
0810 07:53:16.298159 (+ 458us) write_op.cc:270] Start()
0810 07:53:16.298236 (+ 77us) write_op.cc:276] Timestamp: P: 1786348367317984 usec, L: 0
0810 07:53:16.298700 (+ 464us) write_op.cc:270] Start()
0810 07:53:16.298774 (+ 74us) write_op.cc:276] Timestamp: P: 1786348367336182 usec, L: 0
0810 07:53:16.299357 (+ 583us) write_op.cc:270] Start()
0810 07:53:16.299443 (+ 86us) write_op.cc:276] Timestamp: P: 1786348367339672 usec, L: 0
0810 07:53:16.300026 (+ 583us) write_op.cc:270] Start()
0810 07:53:16.300107 (+ 81us) write_op.cc:276] Timestamp: P: 1786348367418686 usec, L: 0
0810 07:53:16.300572 (+ 465us) write_op.cc:270] Start()
0810 07:53:16.300651 (+ 79us) write_op.cc:276] Timestamp: P: 1786348367532591 usec, L: 0
0810 07:53:16.416820 (+116169us) write_op.cc:270] Start()
0810 07:53:16.416965 (+ 145us) write_op.cc:276] Timestamp: P: 1786348367566755 usec, L: 0
0810 07:53:16.417848 (+ 883us) write_op.cc:270] Start()
0810 07:53:16.418010 (+ 162us) write_op.cc:276] Timestamp: P: 1786348367581030 usec, L: 0
0810 07:53:16.418613 (+ 603us) write_op.cc:270] Start()
0810 07:53:16.418706 (+ 93us) write_op.cc:276] Timestamp: P: 1786348367599591 usec, L: 0
0810 07:53:16.419339 (+ 633us) write_op.cc:270] Start()
0810 07:53:16.419443 (+ 104us) write_op.cc:276] Timestamp: P: 1786348367684169 usec, L: 0
0810 07:53:16.420068 (+ 625us) write_op.cc:270] Start()
0810 07:53:16.420155 (+ 87us) write_op.cc:276] Timestamp: P: 1786348367709921 usec, L: 0
0810 07:53:16.507643 (+ 87488us) write_op.cc:270] Start()
0810 07:53:16.507754 (+ 111us) write_op.cc:276] Timestamp: P: 1786348367739542 usec, L: 0
0810 07:53:16.508578 (+ 824us) write_op.cc:270] Start()
0810 07:53:16.508687 (+ 109us) write_op.cc:276] Timestamp: P: 1786348367789959 usec, L: 0
0810 07:53:16.509546 (+ 859us) write_op.cc:270] Start()
0810 07:53:16.509652 (+ 106us) write_op.cc:276] Timestamp: P: 1786348367960381 usec, L: 0
0810 07:53:16.510333 (+ 681us) write_op.cc:270] Start()
0810 07:53:16.510435 (+ 102us) write_op.cc:276] Timestamp: P: 1786348368017924 usec, L: 0
0810 07:53:16.511088 (+ 653us) write_op.cc:270] Start()
0810 07:53:16.511208 (+ 120us) write_op.cc:276] Timestamp: P: 1786348368041251 usec, L: 0
0810 07:53:16.511814 (+ 606us) write_op.cc:270] Start()
0810 07:53:16.511916 (+ 102us) write_op.cc:276] Timestamp: P: 1786348368123932 usec, L: 0
0810 07:53:16.512540 (+ 624us) write_op.cc:270] Start()
0810 07:53:16.512638 (+ 98us) write_op.cc:276] Timestamp: P: 1786348368221957 usec, L: 0
0810 07:53:16.571327 (+ 58689us) write_op.cc:270] Start()
0810 07:53:16.571490 (+ 163us) write_op.cc:276] Timestamp: P: 1786348368263556 usec, L: 0
0810 07:53:16.572426 (+ 936us) write_op.cc:270] Start()
0810 07:53:16.572594 (+ 168us) write_op.cc:276] Timestamp: P: 1786348368388891 usec, L: 0
0810 07:53:16.624150 (+ 51556us) write_op.cc:270] Start()
0810 07:53:16.624275 (+ 125us) write_op.cc:276] Timestamp: P: 1786348368487895 usec, L: 0
0810 07:53:16.628049 (+ 3774us) write_op.cc:270] Start()
0810 07:53:16.628148 (+ 99us) write_op.cc:276] Timestamp: P: 1786348368571842 usec, L: 0
0810 07:53:16.628700 (+ 552us) write_op.cc:270] Start()
0810 07:53:16.628822 (+ 122us) write_op.cc:276] Timestamp: P: 1786348368589450 usec, L: 0
0810 07:53:16.631935 (+ 3113us) write_op.cc:270] Start()
0810 07:53:16.632061 (+ 126us) write_op.cc:276] Timestamp: P: 1786348368591308 usec, L: 0
0810 07:53:16.632621 (+ 560us) write_op.cc:270] Start()
0810 07:53:16.632701 (+ 80us) write_op.cc:276] Timestamp: P: 1786348368613315 usec, L: 0
0810 07:53:16.651119 (+ 18418us) write_op.cc:270] Start()
0810 07:53:16.651262 (+ 143us) write_op.cc:276] Timestamp: P: 1786348368615313 usec, L: 0
0810 07:53:16.652161 (+ 899us) write_op.cc:270] Start()
0810 07:53:16.652294 (+ 133us) write_op.cc:276] Timestamp: P: 1786348368632866 usec, L: 0
0810 07:53:16.652954 (+ 660us) write_op.cc:270] Start()
0810 07:53:16.653039 (+ 85us) write_op.cc:276] Timestamp: P: 1786348368725903 usec, L: 0
0810 07:53:16.653540 (+ 501us) write_op.cc:270] Start()
0810 07:53:16.653624 (+ 84us) write_op.cc:276] Timestamp: P: 1786348368815241 usec, L: 0
0810 07:53:16.654119 (+ 495us) write_op.cc:270] Start()
0810 07:53:16.654201 (+ 82us) write_op.cc:276] Timestamp: P: 1786348368822873 usec, L: 0
0810 07:53:16.679020 (+ 24819us) write_op.cc:270] Start()
0810 07:53:16.679167 (+ 147us) write_op.cc:276] Timestamp: P: 1786348368865653 usec, L: 0
0810 07:53:16.680077 (+ 910us) write_op.cc:270] Start()
0810 07:53:16.680227 (+ 150us) write_op.cc:276] Timestamp: P: 1786348368981783 usec, L: 0
0810 07:53:16.681008 (+ 781us) write_op.cc:270] Start()
0810 07:53:16.681122 (+ 114us) write_op.cc:276] Timestamp: P: 1786348369225903 usec, L: 0
0810 07:53:16.681879 (+ 757us) write_op.cc:270] Start()
0810 07:53:16.682031 (+ 152us) write_op.cc:276] Timestamp: P: 1786348369227855 usec, L: 0
0810 07:53:16.682720 (+ 689us) write_op.cc:270] Start()
0810 07:53:16.682819 (+ 99us) write_op.cc:276] Timestamp: P: 1786348369233282 usec, L: 0
0810 07:53:16.683636 (+ 817us) write_op.cc:270] Start()
0810 07:53:16.683731 (+ 95us) write_op.cc:276] Timestamp: P: 1786348369235642 usec, L: 0
0810 07:53:16.726967 (+ 43236us) write_op.cc:270] Start()
0810 07:53:16.727076 (+ 109us) write_op.cc:276] Timestamp: P: 1786348369237708 usec, L: 0
0810 07:53:16.727741 (+ 665us) write_op.cc:270] Start()
0810 07:53:16.728048 (+ 307us) write_op.cc:276] Timestamp: P: 1786348369239702 usec, L: 0
0810 07:53:16.728782 (+ 734us) write_op.cc:270] Start()
0810 07:53:16.728875 (+ 93us) write_op.cc:276] Timestamp: P: 1786348369247462 usec, L: 0
0810 07:53:16.729485 (+ 610us) write_op.cc:270] Start()
0810 07:53:16.729576 (+ 91us) write_op.cc:276] Timestamp: P: 1786348369328551 usec, L: 0
0810 07:53:16.730155 (+ 579us) write_op.cc:270] Start()
0810 07:53:16.730255 (+ 100us) write_op.cc:276] Timestamp: P: 1786348369343647 usec, L: 0
0810 07:53:16.730801 (+ 546us) write_op.cc:270] Start()
0810 07:53:16.730912 (+ 111us) write_op.cc:276] Timestamp: P: 1786348369569029 usec, L: 0
0810 07:53:16.731454 (+ 542us) write_op.cc:270] Start()
0810 07:53:16.731543 (+ 89us) write_op.cc:276] Timestamp: P: 1786348369714220 usec, L: 0
0810 07:53:16.732083 (+ 540us) write_op.cc:270] Start()
0810 07:53:16.732182 (+ 99us) write_op.cc:276] Timestamp: P: 1786348369722251 usec, L: 0
0810 07:53:16.771314 (+ 39132us) write_op.cc:270] Start()
0810 07:53:16.771445 (+ 131us) write_op.cc:276] Timestamp: P: 1786348369900333 usec, L: 0
0810 07:53:16.772684 (+ 1239us) write_op.cc:270] Start()
0810 07:53:16.772831 (+ 147us) write_op.cc:276] Timestamp: P: 1786348369906644 usec, L: 0
0810 07:53:16.773433 (+ 602us) write_op.cc:270] Start()
0810 07:53:16.773508 (+ 75us) write_op.cc:276] Timestamp: P: 1786348369979726 usec, L: 0
0810 07:53:16.774042 (+ 534us) write_op.cc:270] Start()
0810 07:53:16.774124 (+ 82us) write_op.cc:276] Timestamp: P: 1786348370026948 usec, L: 0
0810 07:53:16.774596 (+ 472us) write_op.cc:270] Start()
0810 07:53:16.774671 (+ 75us) write_op.cc:276] Timestamp: P: 1786348370035447 usec, L: 0
0810 07:53:16.775125 (+ 454us) write_op.cc:270] Start()
0810 07:53:16.775200 (+ 75us) write_op.cc:276] Timestamp: P: 1786348370296430 usec, L: 0
0810 07:53:16.808636 (+ 33436us) write_op.cc:270] Start()
0810 07:53:16.808739 (+ 103us) write_op.cc:276] Timestamp: P: 1786348370465274 usec, L: 0
0810 07:53:16.809219 (+ 480us) write_op.cc:270] Start()
0810 07:53:16.809322 (+ 103us) write_op.cc:276] Timestamp: P: 1786348370576196 usec, L: 0
0810 07:53:16.815510 (+ 6188us) write_op.cc:270] Start()
0810 07:53:16.815952 (+ 442us) write_op.cc:276] Timestamp: P: 1786348370665393 usec, L: 0
0810 07:53:16.816493 (+ 541us) write_op.cc:270] Start()
0810 07:53:16.816569 (+ 76us) write_op.cc:276] Timestamp: P: 1786348370674737 usec, L: 0
0810 07:53:16.817090 (+ 521us) write_op.cc:270] Start()
0810 07:53:16.817176 (+ 86us) write_op.cc:276] Timestamp: P: 1786348370710303 usec, L: 0
0810 07:53:16.817672 (+ 496us) write_op.cc:270] Start()
0810 07:53:16.817763 (+ 91us) write_op.cc:276] Timestamp: P: 1786348370792967 usec, L: 0
0810 07:53:16.818323 (+ 560us) write_op.cc:270] Start()
0810 07:53:16.818419 (+ 96us) write_op.cc:276] Timestamp: P: 1786348370800787 usec, L: 0
0810 07:53:16.818927 (+ 508us) write_op.cc:270] Start()
0810 07:53:16.819000 (+ 73us) write_op.cc:276] Timestamp: P: 1786348370806100 usec, L: 0
0810 07:53:16.819520 (+ 520us) write_op.cc:270] Start()
0810 07:53:16.819591 (+ 71us) write_op.cc:276] Timestamp: P: 1786348370811337 usec, L: 0
0810 07:53:16.820083 (+ 492us) write_op.cc:270] Start()
0810 07:53:16.820169 (+ 86us) write_op.cc:276] Timestamp: P: 1786348370917391 usec, L: 0
0810 07:53:16.820672 (+ 503us) write_op.cc:270] Start()
0810 07:53:16.843316 (+ 22644us) write_op.cc:276] Timestamp: P: 1786348370937987 usec, L: 0
0810 07:53:16.843922 (+ 606us) write_op.cc:270] Start()
0810 07:53:16.844036 (+ 114us) write_op.cc:276] Timestamp: P: 1786348370964349 usec, L: 0
0810 07:53:16.844494 (+ 458us) write_op.cc:270] Start()
0810 07:53:16.844562 (+ 68us) write_op.cc:276] Timestamp: P: 1786348371090628 usec, L: 0
0810 07:53:16.853985 (+ 9423us) write_op.cc:270] Start()
0810 07:53:16.854067 (+ 82us) write_op.cc:276] Timestamp: P: 1786348371216988 usec, L: 0
0810 07:53:16.854510 (+ 443us) write_op.cc:270] Start()
0810 07:53:16.854575 (+ 65us) write_op.cc:276] Timestamp: P: 1786348371418540 usec, L: 0
0810 07:53:16.854980 (+ 405us) write_op.cc:270] Start()
0810 07:53:16.855046 (+ 66us) write_op.cc:276] Timestamp: P: 1786348371704251 usec, L: 0
0810 07:53:16.855511 (+ 465us) write_op.cc:270] Start()
0810 07:53:16.855577 (+ 66us) write_op.cc:276] Timestamp: P: 1786348371859185 usec, L: 0
0810 07:53:16.866383 (+ 10806us) write_op.cc:270] Start()
0810 07:53:16.866471 (+ 88us) write_op.cc:276] Timestamp: P: 1786348371895771 usec, L: 0
0810 07:53:16.886235 (+ 19764us) write_op.cc:270] Start()
0810 07:53:16.886360 (+ 125us) write_op.cc:276] Timestamp: P: 1786348371914253 usec, L: 0
0810 07:53:16.886955 (+ 595us) write_op.cc:270] Start()
0810 07:53:16.887029 (+ 74us) write_op.cc:276] Timestamp: P: 1786348371935650 usec, L: 0
0810 07:53:16.887483 (+ 454us) write_op.cc:270] Start()
0810 07:53:16.887638 (+ 155us) write_op.cc:276] Timestamp: P: 1786348372038681 usec, L: 0
0810 07:53:16.888090 (+ 452us) write_op.cc:270] Start()
0810 07:53:16.888160 (+ 70us) write_op.cc:276] Timestamp: P: 1786348372050313 usec, L: 0
0810 07:53:16.888575 (+ 415us) write_op.cc:270] Start()
0810 07:53:16.888644 (+ 69us) write_op.cc:276] Timestamp: P: 1786348372196098 usec, L: 0
0810 07:53:16.982107 (+ 93463us) write_op.cc:270] Start()
0810 07:53:16.982200 (+ 93us) write_op.cc:276] Timestamp: P: 1786348372242401 usec, L: 0
0810 07:53:16.982749 (+ 549us) write_op.cc:270] Start()
0810 07:53:16.982822 (+ 73us) write_op.cc:276] Timestamp: P: 1786348372244542 usec, L: 0
0810 07:53:16.983258 (+ 436us) write_op.cc:270] Start()
0810 07:53:16.983359 (+ 101us) write_op.cc:276] Timestamp: P: 1786348372297124 usec, L: 0
0810 07:53:16.983783 (+ 424us) write_op.cc:270] Start()
0810 07:53:16.983861 (+ 78us) write_op.cc:276] Timestamp: P: 1786348372498505 usec, L: 0
0810 07:53:16.984294 (+ 433us) write_op.cc:270] Start()
0810 07:53:16.984361 (+ 67us) write_op.cc:276] Timestamp: P: 1786348372500916 usec, L: 0
0810 07:53:16.984833 (+ 472us) write_op.cc:270] Start()
0810 07:53:16.984901 (+ 68us) write_op.cc:276] Timestamp: P: 1786348372505497 usec, L: 0
0810 07:53:16.985351 (+ 450us) write_op.cc:270] Start()
0810 07:53:16.985419 (+ 68us) write_op.cc:276] Timestamp: P: 1786348372507894 usec, L: 0
0810 07:53:16.985834 (+ 415us) write_op.cc:270] Start()
0810 07:53:16.985908 (+ 74us) write_op.cc:276] Timestamp: P: 1786348372726551 usec, L: 0
0810 07:53:16.986351 (+ 443us) write_op.cc:270] Start()
0810 07:53:16.986417 (+ 66us) write_op.cc:276] Timestamp: P: 1786348372816476 usec, L: 0
0810 07:53:16.987801 (+ 1384us) write_op.cc:270] Start()
0810 07:53:16.987871 (+ 70us) write_op.cc:276] Timestamp: P: 1786348372824611 usec, L: 0
0810 07:53:16.988311 (+ 440us) write_op.cc:270] Start()
0810 07:53:16.988377 (+ 66us) write_op.cc:276] Timestamp: P: 1786348373024749 usec, L: 0
0810 07:53:16.988950 (+ 573us) write_op.cc:270] Start()
0810 07:53:16.989030 (+ 80us) write_op.cc:276] Timestamp: P: 1786348373028430 usec, L: 0
0810 07:53:16.989571 (+ 541us) write_op.cc:270] Start()
0810 07:53:16.989634 (+ 63us) write_op.cc:276] Timestamp: P: 1786348373084876 usec, L: 0
0810 07:53:16.990065 (+ 431us) write_op.cc:270] Start()
0810 07:53:16.990133 (+ 68us) write_op.cc:276] Timestamp: P: 1786348373147161 usec, L: 0
0810 07:53:17.018187 (+ 28054us) write_op.cc:270] Start()
0810 07:53:17.018299 (+ 112us) write_op.cc:276] Timestamp: P: 1786348373225899 usec, L: 0
0810 07:53:17.018902 (+ 603us) write_op.cc:270] Start()
0810 07:53:17.018978 (+ 76us) write_op.cc:276] Timestamp: P: 1786348373229406 usec, L: 0
0810 07:53:17.019418 (+ 440us) write_op.cc:270] Start()
0810 07:53:17.019493 (+ 75us) write_op.cc:276] Timestamp: P: 1786348373279850 usec, L: 0
0810 07:53:17.019949 (+ 456us) write_op.cc:270] Start()
0810 07:53:17.020026 (+ 77us) write_op.cc:276] Timestamp: P: 1786348373296815 usec, L: 0
0810 07:53:17.020453 (+ 427us) write_op.cc:270] Start()
0810 07:53:17.020525 (+ 72us) write_op.cc:276] Timestamp: P: 1786348373305604 usec, L: 0
0810 07:53:17.021247 (+ 722us) write_op.cc:270] Start()
0810 07:53:17.021347 (+ 100us) write_op.cc:276] Timestamp: P: 1786348373445059 usec, L: 0
0810 07:53:17.021886 (+ 539us) write_op.cc:270] Start()
0810 07:53:17.021969 (+ 83us) write_op.cc:276] Timestamp: P: 1786348373505420 usec, L: 0
0810 07:53:17.022444 (+ 475us) write_op.cc:270] Start()
0810 07:53:17.022518 (+ 74us) write_op.cc:276] Timestamp: P: 1786348373581851 usec, L: 0
0810 07:53:17.023013 (+ 495us) write_op.cc:270] Start()
0810 07:53:17.023085 (+ 72us) write_op.cc:276] Timestamp: P: 1786348373703678 usec, L: 0
0810 07:53:17.023559 (+ 474us) write_op.cc:270] Start()
0810 07:53:17.023632 (+ 73us) write_op.cc:276] Timestamp: P: 1786348373779768 usec, L: 0
0810 07:53:17.024093 (+ 461us) write_op.cc:270] Start()
0810 07:53:17.024163 (+ 70us) write_op.cc:276] Timestamp: P: 1786348373810209 usec, L: 0
0810 07:53:17.024614 (+ 451us) write_op.cc:270] Start()
0810 07:53:17.024691 (+ 77us) write_op.cc:276] Timestamp: P: 1786348373844624 usec, L: 0
0810 07:53:17.025362 (+ 671us) write_op.cc:270] Start()
0810 07:53:17.025436 (+ 74us) write_op.cc:276] Timestamp: P: 1786348373859345 usec, L: 0
0810 07:53:17.025920 (+ 484us) write_op.cc:270] Start()
0810 07:53:17.026001 (+ 81us) write_op.cc:276] Timestamp: P: 1786348373863711 usec, L: 0
0810 07:53:17.026611 (+ 610us) write_op.cc:270] Start()
0810 07:53:17.026686 (+ 75us) write_op.cc:276] Timestamp: P: 1786348373898405 usec, L: 0
0810 07:53:17.027140 (+ 454us) write_op.cc:270] Start()
0810 07:53:17.027214 (+ 74us) write_op.cc:276] Timestamp: P: 1786348374011623 usec, L: 0
0810 07:53:17.027648 (+ 434us) write_op.cc:270] Start()
0810 07:53:17.027719 (+ 71us) write_op.cc:276] Timestamp: P: 1786348374079526 usec, L: 0
0810 07:53:17.028310 (+ 591us) write_op.cc:270] Start()
0810 07:53:17.028390 (+ 80us) write_op.cc:276] Timestamp: P: 1786348374083571 usec, L: 0
0810 07:53:17.029476 (+ 1086us) write_op.cc:270] Start()
0810 07:53:17.029569 (+ 93us) write_op.cc:276] Timestamp: P: 1786348374571978 usec, L: 0
0810 07:53:17.030121 (+ 552us) write_op.cc:270] Start()
0810 07:53:17.030200 (+ 79us) write_op.cc:276] Timestamp: P: 1786348374602351 usec, L: 0
0810 07:53:17.030657 (+ 457us) write_op.cc:270] Start()
0810 07:53:17.030731 (+ 74us) write_op.cc:276] Timestamp: P: 1786348374643514 usec, L: 0
0810 07:53:17.031171 (+ 440us) write_op.cc:270] Start()
0810 07:53:17.031244 (+ 73us) write_op.cc:276] Timestamp: P: 1786348374695330 usec, L: 0
0810 07:53:17.031695 (+ 451us) write_op.cc:270] Start()
0810 07:53:17.031758 (+ 63us) write_op.cc:276] Timestamp: P: 1786348374803419 usec, L: 0
0810 07:53:17.032166 (+ 408us) write_op.cc:270] Start()
0810 07:53:17.032229 (+ 63us) write_op.cc:276] Timestamp: P: 1786348374859992 usec, L: 0
0810 07:53:17.032644 (+ 415us) write_op.cc:270] Start()
0810 07:53:17.032708 (+ 64us) write_op.cc:276] Timestamp: P: 1786348374978425 usec, L: 0
0810 07:53:17.153459 (+120751us) write_op.cc:270] Start()
0810 07:53:17.153563 (+ 104us) write_op.cc:276] Timestamp: P: 1786348375027490 usec, L: 0
0810 07:53:17.154055 (+ 492us) write_op.cc:270] Start()
0810 07:53:17.154131 (+ 76us) write_op.cc:276] Timestamp: P: 1786348375035598 usec, L: 0
0810 07:53:17.154618 (+ 487us) write_op.cc:270] Start()
0810 07:53:17.154691 (+ 73us) write_op.cc:276] Timestamp: P: 1786348375067093 usec, L: 0
0810 07:53:17.155179 (+ 488us) write_op.cc:270] Start()
0810 07:53:17.155251 (+ 72us) write_op.cc:276] Timestamp: P: 1786348375110940 usec, L: 0
0810 07:53:17.155700 (+ 449us) write_op.cc:270] Start()
0810 07:53:17.155781 (+ 81us) write_op.cc:276] Timestamp: P: 1786348375123572 usec, L: 0
0810 07:53:17.212080 (+ 56299us) write_op.cc:270] Start()
0810 07:53:17.212216 (+ 136us) write_op.cc:276] Timestamp: P: 1786348375235307 usec, L: 0
0810 07:53:17.212875 (+ 659us) write_op.cc:270] Start()
0810 07:53:17.212957 (+ 82us) write_op.cc:276] Timestamp: P: 1786348375290587 usec, L: 0
0810 07:53:17.213464 (+ 507us) write_op.cc:270] Start()
0810 07:53:17.213541 (+ 77us) write_op.cc:276] Timestamp: P: 1786348375360324 usec, L: 0
0810 07:53:17.214002 (+ 461us) write_op.cc:270] Start()
0810 07:53:17.214075 (+ 73us) write_op.cc:276] Timestamp: P: 1786348375382710 usec, L: 0
0810 07:53:17.214609 (+ 534us) write_op.cc:270] Start()
0810 07:53:17.214691 (+ 82us) write_op.cc:276] Timestamp: P: 1786348375389960 usec, L: 0
0810 07:53:17.215221 (+ 530us) write_op.cc:270] Start()
0810 07:53:17.215454 (+ 233us) write_op.cc:276] Timestamp: P: 1786348375420013 usec, L: 0
0810 07:53:17.360765 (+145311us) write_op.cc:270] Start()
0810 07:53:17.360859 (+ 94us) write_op.cc:276] Timestamp: P: 1786348375435824 usec, L: 0
0810 07:53:17.361323 (+ 464us) write_op.cc:270] Start()
0810 07:53:17.361390 (+ 67us) write_op.cc:276] Timestamp: P: 1786348375506228 usec, L: 0
0810 07:53:17.361813 (+ 423us) write_op.cc:270] Start()
0810 07:53:17.361879 (+ 66us) write_op.cc:276] Timestamp: P: 1786348375532854 usec, L: 0
0810 07:53:17.362318 (+ 439us) write_op.cc:270] Start()
0810 07:53:17.362383 (+ 65us) write_op.cc:276] Timestamp: P: 1786348375581778 usec, L: 0
0810 07:53:17.362785 (+ 402us) write_op.cc:270] Start()
0810 07:53:17.362848 (+ 63us) write_op.cc:276] Timestamp: P: 1786348375684208 usec, L: 0
0810 07:53:17.363285 (+ 437us) write_op.cc:270] Start()
0810 07:53:17.363354 (+ 69us) write_op.cc:276] Timestamp: P: 1786348375736658 usec, L: 0
0810 07:53:17.363768 (+ 414us) write_op.cc:270] Start()
0810 07:53:17.363839 (+ 71us) write_op.cc:276] Timestamp: P: 1786348375760181 usec, L: 0
0810 07:53:17.364263 (+ 424us) write_op.cc:270] Start()
0810 07:53:17.364340 (+ 77us) write_op.cc:276] Timestamp: P: 1786348375816137 usec, L: 0
0810 07:53:17.420295 (+ 55955us) write_op.cc:270] Start()
0810 07:53:17.420411 (+ 116us) write_op.cc:276] Timestamp: P: 1786348375824931 usec, L: 0
0810 07:53:17.426945 (+ 6534us) write_op.cc:270] Start()
0810 07:53:17.427040 (+ 95us) write_op.cc:276] Timestamp: P: 1786348375931735 usec, L: 0
0810 07:53:17.427619 (+ 579us) write_op.cc:270] Start()
0810 07:53:17.427702 (+ 83us) write_op.cc:276] Timestamp: P: 1786348375985652 usec, L: 0
0810 07:53:17.428227 (+ 525us) write_op.cc:270] Start()
0810 07:53:17.428305 (+ 78us) write_op.cc:276] Timestamp: P: 1786348376061445 usec, L: 0
0810 07:53:17.445233 (+ 16928us) write_op.cc:270] Start()
0810 07:53:17.445340 (+ 107us) write_op.cc:276] Timestamp: P: 1786348376074099 usec, L: 0
0810 07:53:17.455923 (+ 10583us) write_op.cc:270] Start()
0810 07:53:17.457299 (+ 1376us) write_op.cc:276] Timestamp: P: 1786348
I20260810 07:53:33.439332 677 raft_consensus.cc:2468] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:33.473184 1047 tablet_bootstrap.cc:654] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.489866 1064 tablet_bootstrap.cc:654] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.504048 885 raft_consensus.cc:2468] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:33.599381 1071 raft_consensus.cc:359] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.600138 1071 raft_consensus.cc:385] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.600414 1071 raft_consensus.cc:740] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.606564 1064 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:33.607021 1064 ts_tablet_manager.cc:1403] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.410s user 0.010s sys 0.003s
I20260810 07:53:33.613106 3974 raft_consensus.cc:493] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:33.613601 3974 raft_consensus.cc:515] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.614048 792 leader_election.cc:304] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:33.615767 3974 leader_election.cc:290] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:33.617918 1071 consensus_queue.cc:260] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.589052 407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:33.623364 1047 tablet_bootstrap.cc:492] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:33.623816 1047 ts_tablet_manager.cc:1403] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.236s user 0.020s sys 0.006s
I20260810 07:53:33.625515 3931 raft_consensus.cc:2804] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.626103 3931 raft_consensus.cc:697] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:33.626973 3931 consensus_queue.cc:237] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.627523 1064 raft_consensus.cc:359] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.628568 1064 raft_consensus.cc:385] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.628989 1064 raft_consensus.cc:740] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:33.629762 1064 consensus_queue.cc:260] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.627492 407 raft_consensus.cc:3060] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:33.640496 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:33.641381 417 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:33.669205 1064 ts_tablet_manager.cc:1434] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260810 07:53:33.670485 1064 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:33.672811 3919 raft_consensus.cc:493] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:33.673326 3919 raft_consensus.cc:515] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.677049 3919 leader_election.cc:290] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:33.644977 792 leader_election.cc:343] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.681746 792 leader_election.cc:343] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.682603 792 leader_election.cc:343] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.686868 1047 raft_consensus.cc:359] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.687947 1047 raft_consensus.cc:385] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:33.692878 407 raft_consensus.cc:2468] T 9b4339a26ad540f4b385f69fd0dcdc5d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:33.688247 1047 raft_consensus.cc:740] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
W20260810 07:53:33.692682 792 leader_election.cc:343] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.698699 792 leader_election.cc:343] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.701347 1047 consensus_queue.cc:260] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:33.708904 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 185238us
I20260810 07:53:33.711938 1047 ts_tablet_manager.cc:1434] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.088s user 0.005s sys 0.003s
I20260810 07:53:33.714685 309 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.716418 3912 raft_consensus.cc:2749] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:53:33.765205 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 48216.4us
W20260810 07:53:33.765898 792 leader_election.cc:343] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.766290 792 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.787833 1047 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:33.795893 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 81312.7us
I20260810 07:53:33.797279 309 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.813443 3919 raft_consensus.cc:2749] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:53:33.816243 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 48957.8us
I20260810 07:53:33.818430 1071 ts_tablet_manager.cc:1434] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.427s user 0.004s sys 0.009s
I20260810 07:53:33.819520 1071 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:33.835657 792 leader_election.cc:343] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:33.836335 792 leader_election.cc:304] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:33.848790 1064 tablet_bootstrap.cc:654] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.852411 4011 raft_consensus.cc:2749] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:53:33.853478 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 56281.8us
I20260810 07:53:33.870185 3970 raft_consensus.cc:493] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:33.870921 3919 raft_consensus.cc:2804] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.870756 3970 raft_consensus.cc:515] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.871809 3919 raft_consensus.cc:697] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:33.872752 3919 consensus_queue.cc:237] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.875801 3970 leader_election.cc:290] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.883154 1047 tablet_bootstrap.cc:654] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:33.889552 4008 raft_consensus.cc:493] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:33.891112 4008 raft_consensus.cc:515] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.897861 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.897899 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:33.899521 308 leader_election.cc:343] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:33.900053 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 46027.7us
I20260810 07:53:33.900676 309 leader_election.cc:304] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.901872 4011 raft_consensus.cc:2804] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.902328 4011 raft_consensus.cc:697] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:33.903148 4011 consensus_queue.cc:237] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.905774 4008 leader_election.cc:290] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:33.923925 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:33.977103 4004 consensus_queue.cc:1048] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:33.979533 3919 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:33.980094 3919 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:33.982244 3919 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:33.982239 309 leader_election.cc:304] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:33.982915 4004 raft_consensus.cc:2804] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:33.983325 4004 raft_consensus.cc:697] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:33.984089 4004 consensus_queue.cc:237] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:33.994724 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:33.995496 409 raft_consensus.cc:2393] T 91c2e1bcd13a4c22b2141949b2d73188 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:34.008643 1071 tablet_bootstrap.cc:654] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.016633 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.025241 1064 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:34.025933 1064 ts_tablet_manager.cc:1403] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.356s user 0.011s sys 0.001s
I20260810 07:53:34.028666 1064 raft_consensus.cc:359] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.033672 1064 raft_consensus.cc:385] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.033968 1064 raft_consensus.cc:740] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.034657 1064 consensus_queue.cc:260] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.049844 1064 ts_tablet_manager.cc:1434] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
W20260810 07:53:34.050318 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 49434.6us
I20260810 07:53:34.077700 1064 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:34.082798 1071 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:34.083222 1071 ts_tablet_manager.cc:1403] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.264s user 0.001s sys 0.008s
I20260810 07:53:34.088696 1047 tablet_bootstrap.cc:492] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:34.089147 1047 ts_tablet_manager.cc:1403] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.302s user 0.012s sys 0.003s
I20260810 07:53:34.092710 1047 raft_consensus.cc:359] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.093400 1047 raft_consensus.cc:385] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.093657 1047 raft_consensus.cc:740] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.094326 1047 consensus_queue.cc:260] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.096554 1047 ts_tablet_manager.cc:1434] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260810 07:53:34.110841 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.116297 4011 consensus_queue.cc:1048] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:34.119606 3660 raft_consensus.cc:493] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.120121 3660 raft_consensus.cc:515] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.122375 3660 leader_election.cc:290] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.135982 1071 raft_consensus.cc:359] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.149534 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.151264 1064 tablet_bootstrap.cc:654] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.156838 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1ddea2d9f3462092fcbaee7774cd01" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.157718 890 raft_consensus.cc:2393] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:34.136646 1071 raft_consensus.cc:385] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.162057 1071 raft_consensus.cc:740] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.162762 1071 consensus_queue.cc:260] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.170842 1047 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:34.176227 4011 consensus_queue.cc:1048] T 140522b9e6154f7daca10fdf0c48e6e2 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:34.177019 1071 ts_tablet_manager.cc:1434] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.093s user 0.008s sys 0.000s
I20260810 07:53:34.177942 3970 raft_consensus.cc:493] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.178524 3970 raft_consensus.cc:515] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.181064 3970 leader_election.cc:290] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.215296 1064 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:34.215752 1064 ts_tablet_manager.cc:1403] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.138s user 0.007s sys 0.002s
I20260810 07:53:34.226845 415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.227433 415 raft_consensus.cc:2468] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:34.228119 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.249547 1071 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:34.263401 1047 tablet_bootstrap.cc:654] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.270620 574 leader_election.cc:304] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:34.271341 3970 raft_consensus.cc:3060] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.275117 1071 tablet_bootstrap.cc:654] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.266078 1064 raft_consensus.cc:359] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.281414 1064 raft_consensus.cc:385] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.281697 1064 raft_consensus.cc:740] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:34.295626 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 85436.2us
I20260810 07:53:34.316675 1064 consensus_queue.cc:260] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.318255 1064 ts_tablet_manager.cc:1434] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.102s user 0.004s sys 0.002s
I20260810 07:53:34.319213 1064 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:34.339177 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.340206 417 raft_consensus.cc:2393] T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:34.343156 3970 raft_consensus.cc:2749] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.359020 574 leader_election.cc:304] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:34.359511 3997 raft_consensus.cc:493] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.360025 3997 raft_consensus.cc:515] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.361156 3906 raft_consensus.cc:3060] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.362462 3997 leader_election.cc:290] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.372671 792 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.391076 3906 raft_consensus.cc:2749] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.395810 3934 raft_consensus.cc:2749] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.402191 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:34.403667 308 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:34.405242 309 leader_election.cc:343] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.406358 1047 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:34.406988 1047 ts_tablet_manager.cc:1403] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.236s user 0.011s sys 0.000s
I20260810 07:53:34.418133 1047 raft_consensus.cc:359] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.418820 1047 raft_consensus.cc:385] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.419103 1047 raft_consensus.cc:740] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.419823 1047 consensus_queue.cc:260] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.423167 1047 ts_tablet_manager.cc:1434] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260810 07:53:34.425206 3934 raft_consensus.cc:493] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.425640 3934 raft_consensus.cc:515] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.427507 3934 leader_election.cc:290] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:34.438083 3916 consensus_queue.cc:1048] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:53:34.442018 792 leader_election.cc:343] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.443528 309 leader_election.cc:304] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.445785 3997 raft_consensus.cc:2749] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:34.451409 792 leader_election.cc:343] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.459569 1071 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:34.460462 1071 ts_tablet_manager.cc:1403] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.211s user 0.013s sys 0.000s
W20260810 07:53:34.457993 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:34.480484 3997 consensus_queue.cc:1048] T b6cb01ce3d97454d9d1e7735e51a76c4 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:34.511749 677 raft_consensus.cc:3060] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.515622 1047 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:34.522596 1047 tablet_bootstrap.cc:654] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.523584 4041 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:34.542125 1071 raft_consensus.cc:359] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.542827 1071 raft_consensus.cc:385] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.542640 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:34.543073 1071 raft_consensus.cc:740] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.554215 3916 raft_consensus.cc:493] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.554744 3916 raft_consensus.cc:515] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.556973 3916 leader_election.cc:290] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:34.524128 4041 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:34.565536 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.577322 1047 tablet_bootstrap.cc:492] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:34.577844 1047 ts_tablet_manager.cc:1403] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.062s user 0.012s sys 0.000s
I20260810 07:53:34.580834 4041 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:34.581022 2741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:34.616318 1071 consensus_queue.cc:260] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.618512 1071 ts_tablet_manager.cc:1434] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.157s user 0.006s sys 0.000s
I20260810 07:53:34.619524 1071 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:34.636168 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.580444 1047 raft_consensus.cc:359] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.646147 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:34.646723 1047 raft_consensus.cc:385] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.647068 1047 raft_consensus.cc:740] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.656636 1047 consensus_queue.cc:260] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:34.671705 32451 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 6cd9484084294acfba9613fa49f369b2 (table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6cd9484084294acfba9613fa49f369b2 consensus state has no leader
I20260810 07:53:34.665689 574 leader_election.cc:304] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:34.675940 3660 raft_consensus.cc:2749] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.688747 1047 ts_tablet_manager.cc:1434] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.111s user 0.006s sys 0.000s
W20260810 07:53:34.691918 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:34.700883 792 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.702694 3916 raft_consensus.cc:2749] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:34.711143 792 leader_election.cc:304] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.715898 3916 raft_consensus.cc:2804] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:34.716260 3916 raft_consensus.cc:493] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:34.716580 3916 raft_consensus.cc:3060] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.723308 1071 tablet_bootstrap.cc:654] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.729149 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.729790 403 raft_consensus.cc:2468] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:34.731036 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.737835 3916 raft_consensus.cc:515] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.739837 3916 leader_election.cc:290] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:34.741096 792 leader_election.cc:343] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.741549 792 leader_election.cc:304] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.742704 3934 raft_consensus.cc:2749] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.745692 3934 consensus_queue.cc:1048] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:34.748286 3934 consensus_queue.cc:1048] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:34.759045 309 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.759466 309 leader_election.cc:304] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.760174 4008 raft_consensus.cc:2749] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.761765 1047 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:34.766531 3906 raft_consensus.cc:493] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.767024 3906 raft_consensus.cc:515] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.769687 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.770236 3906 leader_election.cc:290] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.770304 403 raft_consensus.cc:2468] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:34.771486 573 leader_election.cc:304] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:34.773746 3906 raft_consensus.cc:2804] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:34.774163 3906 raft_consensus.cc:493] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:34.774497 3906 raft_consensus.cc:3060] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.778088 3997 raft_consensus.cc:493] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.778628 3997 raft_consensus.cc:515] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.780826 3997 leader_election.cc:290] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.781541 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:34.783386 3934 consensus_queue.cc:1048] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:34.785472 308 leader_election.cc:343] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.786342 894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.786823 792 leader_election.cc:304] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
W20260810 07:53:34.787606 308 consensus_peers.cc:597] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:34.788914 3934 raft_consensus.cc:2804] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:34.789355 3934 raft_consensus.cc:493] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:34.789747 3934 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:34.792703 309 leader_election.cc:343] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.793167 309 leader_election.cc:304] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.795051 4006 raft_consensus.cc:2749] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:34.797058 792 leader_election.cc:343] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.807775 4006 consensus_queue.cc:1048] T e0a74b22a6864180956e763d2b6b6de6 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:34.818825 4055 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:34.820411 1071 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:34.811877 3934 raft_consensus.cc:515] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.820926 1071 ts_tablet_manager.cc:1403] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.202s user 0.011s sys 0.000s
I20260810 07:53:34.823570 1071 raft_consensus.cc:359] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.824226 1071 raft_consensus.cc:385] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:34.825677 3934 leader_election.cc:290] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:34.827457 4055 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:34.839332 1047 tablet_bootstrap.cc:654] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.843076 3906 raft_consensus.cc:515] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:34.845676 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46880us
I20260810 07:53:34.847167 3919 consensus_queue.cc:1048] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:34.849669 3916 consensus_queue.cc:1048] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:34.851543 792 leader_election.cc:343] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.854159 3906 leader_election.cc:290] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.854056 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:34.824549 1071 raft_consensus.cc:740] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:34.854679 403 raft_consensus.cc:3060] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.855358 1071 consensus_queue.cc:260] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:34.852468 792 leader_election.cc:343] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:34.862283 792 leader_election.cc:304] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:34.863538 3916 raft_consensus.cc:2749] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:34.866465 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:34.868711 4055 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:34.886606 1071 ts_tablet_manager.cc:1434] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.065s user 0.006s sys 0.000s
I20260810 07:53:34.890727 403 raft_consensus.cc:2468] T 2905f0597c07439989a00a213fc2194f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:34.892144 573 leader_election.cc:304] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:34.893481 3906 raft_consensus.cc:2804] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:34.896943 1064 tablet_bootstrap.cc:654] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:34.927929 1071 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:34.953807 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:34.954417 403 raft_consensus.cc:3060] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.956125 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:34.905138 3906 raft_consensus.cc:697] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:34.980677 3906 consensus_queue.cc:237] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.982527 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:34.983269 409 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:34.985387 877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:34.984575 4045 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:34.986464 4045 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:34.992069 4045 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:34.993575 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:34.994343 413 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:34.995379 409 raft_consensus.cc:2468] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:34.999544 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:35.004181 888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:35.016387 1047 tablet_bootstrap.cc:492] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:35.016892 1047 ts_tablet_manager.cc:1403] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.255s user 0.013s sys 0.000s
I20260810 07:53:35.018707 4006 consensus_queue.cc:1048] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:35.021166 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:35.021981 886 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
W20260810 07:53:35.028899 574 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:35.031742 1047 raft_consensus.cc:359] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:35.040508 574 leader_election.cc:343] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:35.054242 792 leader_election.cc:304] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:35.055676 4054 raft_consensus.cc:2804] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:35.056131 4054 raft_consensus.cc:697] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:35.057036 4054 consensus_queue.cc:237] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.032511 1047 raft_consensus.cc:385] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.125460 1047 raft_consensus.cc:740] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.127818 1064 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:35.142755 1047 consensus_queue.cc:260] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:35.145574 792 leader_election.cc:343] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:35.146472 792 leader_election.cc:343] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:35.155874 1064 ts_tablet_manager.cc:1403] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.837s user 0.013s sys 0.019s
I20260810 07:53:35.195515 1047 ts_tablet_manager.cc:1434] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.128s user 0.006s sys 0.000s
I20260810 07:53:35.203323 3919 consensus_queue.cc:1048] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:35.222877 4006 consensus_queue.cc:1048] T 5c1ddea2d9f3462092fcbaee7774cd01 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:35.225034 32563 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.643s (SENT) [suppressed 1 similar messages]
W20260810 07:53:35.243093 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 95809.3us
I20260810 07:53:35.240643 1047 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:35.276546 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 148633us
W20260810 07:53:35.278790 4073 log.cc:927] Time spent T e7529f4ac9fb457ba1b3c3bc90eb275d P 60781ef3c8b84a8788f43d77e1bb3b19: Append to log took a long time: real 0.184s user 0.000s sys 0.000s
I20260810 07:53:35.290230 403 raft_consensus.cc:2468] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:35.303004 1071 tablet_bootstrap.cc:654] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:35.310039 1064 raft_consensus.cc:359] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.310874 1064 raft_consensus.cc:385] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.311187 1064 raft_consensus.cc:740] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.311874 1064 consensus_queue.cc:260] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:35.319190 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 66233.4us
W20260810 07:53:35.339880 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 114864us
I20260810 07:53:35.356688 574 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:35.357936 3906 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:35.361598 1047 tablet_bootstrap.cc:654] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:35.365649 3906 raft_consensus.cc:2749] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:35.421861 1064 ts_tablet_manager.cc:1434] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.266s user 0.006s sys 0.001s
I20260810 07:53:35.422976 1064 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:35.429587 1047 tablet_bootstrap.cc:492] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:35.473505 1047 ts_tablet_manager.cc:1403] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.233s user 0.011s sys 0.000s
I20260810 07:53:35.511435 1071 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:35.511948 1071 ts_tablet_manager.cc:1403] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.624s user 0.009s sys 0.004s
I20260810 07:53:35.514561 1071 raft_consensus.cc:359] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.515421 1071 raft_consensus.cc:385] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.515733 1071 raft_consensus.cc:740] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.516533 1071 consensus_queue.cc:260] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.519696 1047 raft_consensus.cc:359] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.520340 1047 raft_consensus.cc:385] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.520581 1047 raft_consensus.cc:740] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.522642 1047 consensus_queue.cc:260] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.524807 1047 ts_tablet_manager.cc:1434] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
W20260810 07:53:35.528550 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 65484.2us
I20260810 07:53:35.535771 1064 tablet_bootstrap.cc:654] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:35.564173 1047 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:35.622071 1071 ts_tablet_manager.cc:1434] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.110s user 0.004s sys 0.000s
I20260810 07:53:35.600960 1047 tablet_bootstrap.cc:654] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:35.651525 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 68775.8us
I20260810 07:53:35.654148 1071 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:35.682981 1064 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:35.683435 1064 ts_tablet_manager.cc:1403] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.261s user 0.012s sys 0.001s
I20260810 07:53:35.710587 1047 tablet_bootstrap.cc:492] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:35.711063 1047 ts_tablet_manager.cc:1403] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.147s user 0.006s sys 0.006s
I20260810 07:53:35.717650 1047 raft_consensus.cc:359] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.719055 1047 raft_consensus.cc:385] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.719383 1047 raft_consensus.cc:740] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.727650 4096 raft_consensus.cc:493] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:35.728333 4096 raft_consensus.cc:515] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.739916 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:35.741546 4096 leader_election.cc:290] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:35.741751 308 leader_election.cc:343] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:35.743818 1064 raft_consensus.cc:359] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.744495 1064 raft_consensus.cc:385] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:35.744769 1064 raft_consensus.cc:740] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:35.745352 1064 consensus_queue.cc:260] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.720077 1047 consensus_queue.cc:260] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.763221 3906 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:35.764107 3906 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.766909 3906 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:35.769189 1047 ts_tablet_manager.cc:1434] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.058s user 0.006s sys 0.002s
I20260810 07:53:35.785429 1064 ts_tablet_manager.cc:1434] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.102s user 0.007s sys 0.000s
I20260810 07:53:35.800827 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:35.814924 417 raft_consensus.cc:2393] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:35.833468 1047 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:35.841439 1071 tablet_bootstrap.cc:654] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:35.884217 1047 tablet_bootstrap.cc:654] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:35.890489 32455 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet c43aeafc19a240e6991b69249b0516fa (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c43aeafc19a240e6991b69249b0516fa consensus state has no leader
I20260810 07:53:35.893797 3906 consensus_queue.cc:1048] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:35.918876 3906 raft_consensus.cc:493] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:35.919407 3906 raft_consensus.cc:515] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:35.922523 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:35.925443 3906 leader_election.cc:290] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:35.944082 1047 tablet_bootstrap.cc:492] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:35.944527 1047 ts_tablet_manager.cc:1403] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.111s user 0.007s sys 0.010s
I20260810 07:53:35.960826 1071 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:35.970990 792 leader_election.cc:304] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:35.979267 891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:35.980470 891 raft_consensus.cc:2468] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:35.985383 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.643s (SENT)
I20260810 07:53:35.985994 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:36.017144 4054 raft_consensus.cc:2804] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:36.017786 4054 raft_consensus.cc:697] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:36.018870 4054 consensus_queue.cc:237] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.021085 3919 consensus_queue.cc:1048] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:36.022742 3919 consensus_queue.cc:1048] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:36.026968 3660 raft_consensus.cc:493] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.027503 3660 raft_consensus.cc:515] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.030242 3660 leader_election.cc:290] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:36.034869 1064 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:36.060264 1064 tablet_bootstrap.cc:654] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.007174 309 leader_election.cc:304] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:36.070292 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 63206.3us
I20260810 07:53:36.007761 3997 raft_consensus.cc:493] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.071264 3997 raft_consensus.cc:515] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.073268 3997 leader_election.cc:290] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:36.074922 4086 raft_consensus.cc:2804] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:36.075299 4086 raft_consensus.cc:493] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:36.075642 4086 raft_consensus.cc:3060] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.037529 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:36.083472 409 raft_consensus.cc:2393] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:36.084587 1047 raft_consensus.cc:359] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.087641 1047 raft_consensus.cc:385] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.087930 1047 raft_consensus.cc:740] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.088572 1047 consensus_queue.cc:260] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.090108 1047 ts_tablet_manager.cc:1434] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.135s user 0.005s sys 0.000s
I20260810 07:53:36.094990 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:36.096596 308 leader_election.cc:343] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:36.101444 4086 raft_consensus.cc:515] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.104791 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:36.106341 308 leader_election.cc:343] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:36.118074 1071 ts_tablet_manager.cc:1403] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.464s user 0.013s sys 0.000s
I20260810 07:53:36.122011 1071 raft_consensus.cc:359] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.122604 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.123402 886 raft_consensus.cc:2468] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:36.147347 1064 tablet_bootstrap.cc:492] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:36.147987 1064 ts_tablet_manager.cc:1403] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.114s user 0.010s sys 0.003s
I20260810 07:53:36.152637 1064 raft_consensus.cc:359] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.165899 1064 raft_consensus.cc:385] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.166406 1064 raft_consensus.cc:740] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.169231 1064 consensus_queue.cc:260] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.173679 1064 ts_tablet_manager.cc:1434] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260810 07:53:36.197301 1071 raft_consensus.cc:385] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.190796 309 leader_election.cc:304] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:36.201742 4004 raft_consensus.cc:2804] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:36.202306 4004 raft_consensus.cc:493] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:36.202729 4004 raft_consensus.cc:3060] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.153256 4086 leader_election.cc:290] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:36.209429 1047 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:36.212116 4004 raft_consensus.cc:515] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.126166 4129 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:36.148986 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 163776us
I20260810 07:53:36.214620 1071 raft_consensus.cc:740] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.225319 1071 consensus_queue.cc:260] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.230870 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:36.231433 886 raft_consensus.cc:3060] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.254639 886 raft_consensus.cc:2468] T f6cd26a04d3c417d87091f05bc5cb5cd P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:36.256328 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 86184.5us
I20260810 07:53:36.274250 1071 ts_tablet_manager.cc:1434] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.155s user 0.006s sys 0.000s
I20260810 07:53:36.279556 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83]
I20260810 07:53:36.281282 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]
I20260810 07:53:36.282582 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97]
I20260810 07:53:36.283885 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250]
I20260810 07:53:36.308444 3934 raft_consensus.cc:493] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.309011 3934 raft_consensus.cc:515] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.311043 3934 leader_election.cc:290] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:36.316344 1047 tablet_bootstrap.cc:654] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.329262 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:36.329945 886 raft_consensus.cc:3060] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.265436 4004 leader_election.cc:290] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:36.345686 1071 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:36.353665 309 leader_election.cc:304] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:36.355297 4092 raft_consensus.cc:2804] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:36.369298 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]
I20260810 07:53:36.370896 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]
I20260810 07:53:36.317637 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
W20260810 07:53:36.318886 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 60214.1us
I20260810 07:53:36.385749 4092 raft_consensus.cc:697] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:36.386680 4092 consensus_queue.cc:237] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.413061 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]
I20260810 07:53:36.414796 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b041641ad99146e8ac57b26f842b8976 [id=0ab295f37b5e482482053f67fe0719a6]
W20260810 07:53:36.420420 2560 leader_election.cc:336] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.745s (SENT)
I20260810 07:53:36.420943 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:36.448202 886 raft_consensus.cc:2468] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:36.458032 308 leader_election.cc:343] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:36.458345 4159 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:36.463968 1071 tablet_bootstrap.cc:654] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:36.455461 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 60318.7us
I20260810 07:53:36.470713 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c]
I20260810 07:53:36.472182 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6]
I20260810 07:53:36.479940 309 leader_election.cc:304] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:36.485591 1064 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:36.488480 1047 tablet_bootstrap.cc:492] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:36.488966 1047 ts_tablet_manager.cc:1403] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.286s user 0.010s sys 0.002s
I20260810 07:53:36.501874 4153 raft_consensus.cc:2804] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:36.502488 4153 raft_consensus.cc:697] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:36.503392 4153 consensus_queue.cc:237] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.505805 1047 raft_consensus.cc:359] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.506529 1047 raft_consensus.cc:385] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.506847 1047 raft_consensus.cc:740] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.516698 1064 tablet_bootstrap.cc:654] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.528818 1047 consensus_queue.cc:260] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.536118 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]
I20260810 07:53:36.539479 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]
I20260810 07:53:36.544313 1071 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:36.544895 1071 ts_tablet_manager.cc:1403] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.200s user 0.012s sys 0.000s
I20260810 07:53:36.547513 1071 raft_consensus.cc:359] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.548259 1071 raft_consensus.cc:385] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.548590 1071 raft_consensus.cc:740] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.597755 1064 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:36.598299 1064 ts_tablet_manager.cc:1403] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.113s user 0.012s sys 0.002s
I20260810 07:53:36.602885 3916 consensus_queue.cc:1048] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:36.613057 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6]
I20260810 07:53:36.614946 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017]
I20260810 07:53:36.616590 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e]
I20260810 07:53:36.617520 1071 consensus_queue.cc:260] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.619086 1071 ts_tablet_manager.cc:1434] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.074s user 0.005s sys 0.000s
I20260810 07:53:36.619061 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f]
W20260810 07:53:36.620074 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 60453.7us
I20260810 07:53:36.634335 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4]
I20260810 07:53:36.636073 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]
I20260810 07:53:36.637689 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]
I20260810 07:53:36.645222 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf]
I20260810 07:53:36.646893 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]
I20260810 07:53:36.649986 1071 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:36.648547 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]
I20260810 07:53:36.651124 1064 raft_consensus.cc:359] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.651791 1064 raft_consensus.cc:385] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.652071 1064 raft_consensus.cc:740] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.652971 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]
I20260810 07:53:36.655470 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43]
I20260810 07:53:36.665475 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9]
I20260810 07:53:36.667265 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]
I20260810 07:53:36.668877 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408]
I20260810 07:53:36.673441 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27]
I20260810 07:53:36.675015 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186]
I20260810 07:53:36.677449 1047 ts_tablet_manager.cc:1434] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.188s user 0.006s sys 0.001s
I20260810 07:53:36.678164 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.679210 886 raft_consensus.cc:2393] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:36.683470 892 raft_consensus.cc:3060] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.689031 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a]
I20260810 07:53:36.690750 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf]
I20260810 07:53:36.692382 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855]
I20260810 07:53:36.694011 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]
I20260810 07:53:36.695569 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]
W20260810 07:53:36.709309 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 47251.9us
I20260810 07:53:36.717195 1064 consensus_queue.cc:260] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.719763 1064 ts_tablet_manager.cc:1434] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.121s user 0.005s sys 0.001s
I20260810 07:53:36.720468 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]
I20260810 07:53:36.722189 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30]
I20260810 07:53:36.723834 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb]
I20260810 07:53:36.726167 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c]
I20260810 07:53:36.727555 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4]
I20260810 07:53:36.728333 1047 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:36.729063 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5]
I20260810 07:53:36.730669 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812]
I20260810 07:53:36.732323 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]
I20260810 07:53:36.733280 670 raft_consensus.cc:3060] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.737574 1064 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:36.743366 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.757241 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.761268 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13]
I20260810 07:53:36.762863 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18]
I20260810 07:53:36.763931 1071 tablet_bootstrap.cc:700] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent opening tablet: real 0.109s user 0.002s sys 0.001s
I20260810 07:53:36.764357 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]
I20260810 07:53:36.764900 1071 tablet_bootstrap.cc:654] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.772748 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]
I20260810 07:53:36.774309 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866]
I20260810 07:53:36.788908 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb]
I20260810 07:53:36.790544 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1]
I20260810 07:53:36.796399 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015]
I20260810 07:53:36.797994 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9]
I20260810 07:53:36.799430 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133]
I20260810 07:53:36.801777 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614]
I20260810 07:53:36.803288 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f]
I20260810 07:53:36.806351 3997 raft_consensus.cc:493] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.807061 3997 raft_consensus.cc:515] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.809124 3997 leader_election.cc:290] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:36.810639 4137 raft_consensus.cc:493] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.811385 4137 raft_consensus.cc:515] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.813045 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898]
I20260810 07:53:36.814482 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0]
I20260810 07:53:36.815227 4137 leader_election.cc:290] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:36.816926 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 68115.4us
I20260810 07:53:36.825253 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:36.826042 417 raft_consensus.cc:2468] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:36.830796 4054 raft_consensus.cc:493] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.831238 4054 raft_consensus.cc:515] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.833117 4054 leader_election.cc:290] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:36.843303 1064 tablet_bootstrap.cc:654] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.847679 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.853061 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:36.866930 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23]
I20260810 07:53:36.868299 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45]
I20260810 07:53:36.873411 4170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:36.873777 1047 tablet_bootstrap.cc:654] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:36.874155 4170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:36.879392 573 leader_election.cc:304] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:36.880928 4148 raft_consensus.cc:2804] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:36.881335 4148 raft_consensus.cc:493] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:36.881651 4148 raft_consensus.cc:3060] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:36.883071 4170 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:36.886102 2741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
W20260810 07:53:36.897214 309 leader_election.cc:343] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:36.901276 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]
W20260810 07:53:36.901332 308 leader_election.cc:343] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:36.901744 308 leader_election.cc:304] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:36.902818 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]
I20260810 07:53:36.903081 4153 raft_consensus.cc:2749] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:36.908842 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:36.909689 886 raft_consensus.cc:2468] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:36.915328 1071 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:36.915789 1071 ts_tablet_manager.cc:1403] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.266s user 0.011s sys 0.001s
I20260810 07:53:36.920830 1071 raft_consensus.cc:359] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.921514 1071 raft_consensus.cc:385] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.921792 1071 raft_consensus.cc:740] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.929325 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d]
I20260810 07:53:36.930670 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d]
W20260810 07:53:36.935529 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56210.6us
I20260810 07:53:36.942817 1064 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:36.943395 1064 ts_tablet_manager.cc:1403] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.206s user 0.008s sys 0.004s
W20260810 07:53:36.943761 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 45523.2us
I20260810 07:53:36.932704 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb]
I20260810 07:53:36.946420 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e]
I20260810 07:53:36.947096 1071 consensus_queue.cc:260] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.948572 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4]
I20260810 07:53:36.949936 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]
I20260810 07:53:36.951385 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853]
I20260810 07:53:36.952803 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]
I20260810 07:53:36.954123 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1]
I20260810 07:53:36.955399 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4]
I20260810 07:53:36.956676 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b]
I20260810 07:53:36.958114 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]
I20260810 07:53:36.959419 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141]
I20260810 07:53:36.973801 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]
I20260810 07:53:36.975255 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264]
W20260810 07:53:36.985176 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 43508.6us
I20260810 07:53:36.985235 1064 raft_consensus.cc:359] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:36.986111 1064 raft_consensus.cc:385] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:36.986433 1064 raft_consensus.cc:740] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:36.991041 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]
I20260810 07:53:36.992424 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a]
I20260810 07:53:36.996464 4042 raft_consensus.cc:493] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:36.997529 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]
I20260810 07:53:36.997131 4042 raft_consensus.cc:515] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:36.998860 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a]
I20260810 07:53:36.999890 1071 ts_tablet_manager.cc:1434] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260810 07:53:37.000166 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f]
I20260810 07:53:37.000304 4042 leader_election.cc:290] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.001565 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7]
I20260810 07:53:37.005152 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d]
I20260810 07:53:37.004961 1064 consensus_queue.cc:260] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.007863 1064 ts_tablet_manager.cc:1434] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.064s user 0.005s sys 0.000s
I20260810 07:53:37.009976 3997 raft_consensus.cc:493] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.010118 4148 raft_consensus.cc:515] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.010625 3997 raft_consensus.cc:515] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.012878 3997 leader_election.cc:290] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.023444 1071 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:37.028304 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793]
I20260810 07:53:37.028638 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:37.034478 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653]
I20260810 07:53:37.036063 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]
I20260810 07:53:37.039952 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]
I20260810 07:53:37.047636 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a]
I20260810 07:53:37.047693 4054 raft_consensus.cc:493] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.049391 4054 raft_consensus.cc:515] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.053660 4054 leader_election.cc:290] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:37.054407 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b3adcec668f7473687d358c9b80c10f4 [id=b23efee8dd99419d9308ef2d5a907409]
I20260810 07:53:37.065186 409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:37.065809 409 raft_consensus.cc:2468] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:37.065898 1047 tablet_bootstrap.cc:492] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:37.066047 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d]
I20260810 07:53:37.066330 1047 ts_tablet_manager.cc:1403] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.338s user 0.010s sys 0.000s
I20260810 07:53:37.067513 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb]
I20260810 07:53:37.069801 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]
I20260810 07:53:37.071326 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257]
I20260810 07:53:37.072754 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b]
I20260810 07:53:37.080193 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d]
I20260810 07:53:37.081722 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72]
I20260810 07:53:37.084328 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.086340 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb]
I20260810 07:53:37.097554 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]
I20260810 07:53:37.106041 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9]
I20260810 07:53:37.108997 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b]
I20260810 07:53:37.108951 413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:37.109644 413 raft_consensus.cc:3060] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:37.114346 670 raft_consensus.cc:2468] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:37.116261 1071 tablet_bootstrap.cc:654] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.118563 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a]
I20260810 07:53:37.116974 413 raft_consensus.cc:2468] T 8ee78955a68a451f81e01a62a57aa4a4 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:37.124275 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e]
I20260810 07:53:37.128980 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]
I20260810 07:53:37.130523 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]
W20260810 07:53:37.130539 309 leader_election.cc:343] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:37.123100 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.134653 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24]
I20260810 07:53:37.137028 1064 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:37.138470 1071 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:37.139036 1071 ts_tablet_manager.cc:1403] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.116s user 0.015s sys 0.001s
I20260810 07:53:37.143723 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079]
I20260810 07:53:37.139675 4148 leader_election.cc:290] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.091096 1047 raft_consensus.cc:359] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.148916 1047 raft_consensus.cc:385] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.161947 1047 raft_consensus.cc:740] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.162135 1071 raft_consensus.cc:359] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.162868 1071 raft_consensus.cc:385] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.163142 1071 raft_consensus.cc:740] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.163376 1047 consensus_queue.cc:260] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.163765 1071 consensus_queue.cc:260] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.165494 1071 ts_tablet_manager.cc:1434] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260810 07:53:37.171829 4185 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:37.175287 1071 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:37.193370 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5]
I20260810 07:53:37.195127 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446]
I20260810 07:53:37.172360 4185 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
W20260810 07:53:37.174153 308 leader_election.cc:343] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:37.212002 308 leader_election.cc:304] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:37.213186 1064 tablet_bootstrap.cc:654] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.213644 4153 raft_consensus.cc:2749] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:37.196571 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e]
I20260810 07:53:37.241434 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961]
I20260810 07:53:37.242825 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f]
I20260810 07:53:37.242780 3997 raft_consensus.cc:493] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:37.243502 3997 raft_consensus.cc:515] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.246667 3997 leader_election.cc:290] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.242177 1071 tablet_bootstrap.cc:654] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.265084 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:37.265801 412 raft_consensus.cc:2468] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
W20260810 07:53:37.225289 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 44197.8us
I20260810 07:53:37.271106 573 leader_election.cc:304] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:37.274053 4181 raft_consensus.cc:2804] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:37.274417 4181 raft_consensus.cc:493] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:37.274729 4181 raft_consensus.cc:3060] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:37.288100 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:37.295493 4185 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:37.299702 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.302032 1047 ts_tablet_manager.cc:1434] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.235s user 0.007s sys 0.000s
I20260810 07:53:37.306865 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]
I20260810 07:53:37.308427 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]
I20260810 07:53:37.312800 574 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
W20260810 07:53:37.327994 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 56101.6us
I20260810 07:53:37.328962 573 leader_election.cc:304] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:37.309933 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279]
I20260810 07:53:37.353799 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957]
I20260810 07:53:37.355791 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]
I20260810 07:53:37.364918 3997 raft_consensus.cc:493] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.365504 3997 raft_consensus.cc:515] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.368105 3997 leader_election.cc:290] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.380057 1071 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:37.380600 1071 ts_tablet_manager.cc:1403] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.206s user 0.038s sys 0.032s
I20260810 07:53:37.382591 4138 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:37.383410 1071 raft_consensus.cc:359] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.384132 1071 raft_consensus.cc:385] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.384397 1071 raft_consensus.cc:740] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.385105 1071 consensus_queue.cc:260] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.389993 4191 consensus_queue.cc:1048] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:37.394397 1071 ts_tablet_manager.cc:1434] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260810 07:53:37.395187 4138 raft_consensus.cc:2749] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:37.414392 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]
I20260810 07:53:37.415722 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168]
I20260810 07:53:37.444237 1064 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:37.445820 1064 ts_tablet_manager.cc:1403] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.437s user 0.016s sys 0.003s
I20260810 07:53:37.447360 3039 catalog_manager.cc:679] Loaded metadata for table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]
W20260810 07:53:37.401119 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 72259.2us
I20260810 07:53:37.401835 4189 raft_consensus.cc:2804] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:53:37.402568 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 87921.7us
I20260810 07:53:37.406965 677 raft_consensus.cc:3060] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:37.408499 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.417589 1047 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:37.432930 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:37.469745 32563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 46947us
I20260810 07:53:37.484118 1064 raft_consensus.cc:359] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.485004 1064 raft_consensus.cc:385] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.485332 1064 raft_consensus.cc:740] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.486359 1064 consensus_queue.cc:260] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.489873 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed74e441397344e09e924fb724ad55d6" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:37.522948 1064 ts_tablet_manager.cc:1434] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.077s user 0.004s sys 0.003s
I20260810 07:53:37.523795 4054 raft_consensus.cc:493] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.524564 4054 raft_consensus.cc:515] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.527470 4054 leader_election.cc:290] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:37.529465 671 raft_consensus.cc:3060] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:37.584508 1071 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:37.591228 1071 tablet_bootstrap.cc:654] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.594704 4189 raft_consensus.cc:697] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:37.599686 4096 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.600503 4096 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.604110 4096 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:37.607208 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 142237us
I20260810 07:53:37.610551 3039 tablet_loader.cc:96] loaded metadata for tablet 01d24209ef6b4c919ce12a702a318577 (table default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1])
I20260810 07:53:37.611481 574 leader_election.cc:304] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:37.612754 3039 tablet_loader.cc:96] loaded metadata for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262])
I20260810 07:53:37.614012 3039 tablet_loader.cc:96] loaded metadata for tablet 05ef1be8eaa84e6bb5586e68268799a0 (table default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a])
I20260810 07:53:37.615223 3039 tablet_loader.cc:96] loaded metadata for tablet 07fe1a943230432dbe533e7b3671172b (table default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb])
I20260810 07:53:37.616235 4194 raft_consensus.cc:2749] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:37.598369 4189 consensus_queue.cc:237] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.619354 574 leader_election.cc:304] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:37.620674 4137 raft_consensus.cc:2749] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:53:37.631120 3039 tablet_loader.cc:96] loaded metadata for tablet 0d3aaf0099934d1e9bede4687b36ce44 (table default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a])
I20260810 07:53:37.631670 4194 raft_consensus.cc:493] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:37.632484 3039 tablet_loader.cc:96] loaded metadata for tablet 0d7084712ff04ac48a54171534e04d69 (table default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb])
I20260810 07:53:37.632609 4194 raft_consensus.cc:515] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.637408 4194 leader_election.cc:290] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.641386 3039 tablet_loader.cc:96] loaded metadata for tablet 0dbe1a6581374213be7c3f63a7a567a4 (table default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186])
I20260810 07:53:37.642729 3039 tablet_loader.cc:96] loaded metadata for tablet 0e15de89beda46f29a650f93f01d4fda (table default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a])
I20260810 07:53:37.644037 3039 tablet_loader.cc:96] loaded metadata for tablet 0e5a69fed75047f18522bfee894c769b (table default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf])
I20260810 07:53:37.646652 3039 tablet_loader.cc:96] loaded metadata for tablet 10341e269a934612bb0cf6ef29818a39 (table default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13])
I20260810 07:53:37.648393 3039 tablet_loader.cc:96] loaded metadata for tablet 140522b9e6154f7daca10fdf0c48e6e2 (table default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961])
W20260810 07:53:37.651569 308 leader_election.cc:343] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:37.573518 1064 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:37.662148 3039 tablet_loader.cc:96] loaded metadata for tablet 1cb66183211f45d4a756f70c5e331f87 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5])
I20260810 07:53:37.663512 3039 tablet_loader.cc:96] loaded metadata for tablet 1d25b4b974034eea9f1256fb4e2a503c (table default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f])
I20260810 07:53:37.664754 3039 tablet_loader.cc:96] loaded metadata for tablet 1e629e89eb734ade82b3dcbf11c971f8 (table default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30])
I20260810 07:53:37.665966 3039 tablet_loader.cc:96] loaded metadata for tablet 1f8941c49c0a4fba9f7de2aabb790901 (table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432])
I20260810 07:53:37.674360 3039 tablet_loader.cc:96] loaded metadata for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015])
I20260810 07:53:37.676900 3039 tablet_loader.cc:96] loaded metadata for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e])
I20260810 07:53:37.679174 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:37.679774 886 raft_consensus.cc:2468] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:37.686772 3039 tablet_loader.cc:96] loaded metadata for tablet 2905f0597c07439989a00a213fc2194f (table default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286])
I20260810 07:53:37.687680 4206 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.688200 4206 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.690301 4206 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:37.694439 1064 tablet_bootstrap.cc:654] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.695533 3039 tablet_loader.cc:96] loaded metadata for tablet 2a91af48fe6a4bd986b5266f09fad6b2 (table default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb])
I20260810 07:53:37.698004 3039 tablet_loader.cc:96] loaded metadata for tablet 2abec82a68a54082afa1d8e72aba6617 (table default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72])
I20260810 07:53:37.699121 3039 tablet_loader.cc:96] loaded metadata for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a (table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4])
I20260810 07:53:37.703271 1071 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:37.703763 1071 ts_tablet_manager.cc:1403] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.120s user 0.010s sys 0.005s
I20260810 07:53:37.706683 4096 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.707265 4096 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.709263 4096 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.713312 4205 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.713956 4205 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.716114 4205 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:37.682260 4181 raft_consensus.cc:515] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.732178 4181 leader_election.cc:290] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.734158 3039 tablet_loader.cc:96] loaded metadata for tablet 2c49060c69f2434db4e9f83f2147d7d1 (table default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d])
I20260810 07:53:37.742856 1047 tablet_bootstrap.cc:654] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.741925 3039 tablet_loader.cc:96] loaded metadata for tablet 38f8faef6d284c98bf5fd9f7c6f9c443 (table default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d])
I20260810 07:53:37.750837 3039 tablet_loader.cc:96] loaded metadata for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d (table default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141])
I20260810 07:53:37.752326 3039 tablet_loader.cc:96] loaded metadata for tablet 3c97418290e04f60884c721ded14495a (table default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653])
I20260810 07:53:37.753751 3039 tablet_loader.cc:96] loaded metadata for tablet 3dfa1b6f604c452e9f346ecc18b5b691 (table default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d])
I20260810 07:53:37.764635 3039 tablet_loader.cc:96] loaded metadata for tablet 433404aa29574d819371fe090e670095 (table default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9])
I20260810 07:53:37.765913 3039 tablet_loader.cc:96] loaded metadata for tablet 446ed19d914548ea864a43b2bb8c7f5a (table default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb])
I20260810 07:53:37.768097 3039 tablet_loader.cc:96] loaded metadata for tablet 44ff256b314e479d981333d9cbfc1656 (table default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b])
I20260810 07:53:37.776252 1064 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:37.778549 1064 ts_tablet_manager.cc:1403] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.205s user 0.002s sys 0.016s
I20260810 07:53:37.787771 3039 tablet_loader.cc:96] loaded metadata for tablet 45a3fe9686e0445187c77cf41c563c01 (table default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb])
W20260810 07:53:37.791456 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 43602.4us
I20260810 07:53:37.793905 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:37.793704 1064 raft_consensus.cc:359] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.794834 403 raft_consensus.cc:2468] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:37.795120 1064 raft_consensus.cc:385] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.795672 1064 raft_consensus.cc:740] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.798174 1064 consensus_queue.cc:260] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.805094 3039 tablet_loader.cc:96] loaded metadata for tablet 461d70a71f2c44228f4220e7f5fb9c64 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279])
I20260810 07:53:37.805038 1064 ts_tablet_manager.cc:1434] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.007s sys 0.001s
I20260810 07:53:37.807286 3039 tablet_loader.cc:96] loaded metadata for tablet 4bb34e5b988b497c8f72a6c7266c0f2d (table default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c])
I20260810 07:53:37.810566 1064 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:37.837651 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 49539.4us
I20260810 07:53:37.840507 3039 tablet_loader.cc:96] loaded metadata for tablet 4d6df1791e5146a38f3cf0304b5034cb (table default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079])
I20260810 07:53:37.851106 1064 tablet_bootstrap.cc:654] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.863883 1047 tablet_bootstrap.cc:492] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:37.864315 1047 ts_tablet_manager.cc:1403] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.447s user 0.013s sys 0.003s
I20260810 07:53:37.864125 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.864813 671 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:37.867527 1047 raft_consensus.cc:359] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.869715 1047 raft_consensus.cc:385] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.870024 1047 raft_consensus.cc:740] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.870795 1047 consensus_queue.cc:260] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.872982 4004 consensus_queue.cc:1048] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:37.873751 3039 tablet_loader.cc:96] loaded metadata for tablet 4ecc6870602043269daa61cb173108b3 (table default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b])
I20260810 07:53:37.875197 3039 tablet_loader.cc:96] loaded metadata for tablet 4fb366297938415382cf6555ed052956 (table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151])
I20260810 07:53:37.876271 3039 tablet_loader.cc:96] loaded metadata for tablet 505e2f7a25a447c3bf19b310aca1702c (table default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e])
I20260810 07:53:37.879889 3039 tablet_loader.cc:96] loaded metadata for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc])
I20260810 07:53:37.881856 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
W20260810 07:53:37.883533 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 66797.2us
I20260810 07:53:37.887046 1047 ts_tablet_manager.cc:1434] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.003s sys 0.006s
I20260810 07:53:37.887833 1064 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:37.888168 1047 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:37.888427 1064 ts_tablet_manager.cc:1403] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.078s user 0.007s sys 0.011s
I20260810 07:53:37.897895 1071 raft_consensus.cc:359] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:37.897958 2560 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.689s (SENT) [suppressed 1 similar messages]
I20260810 07:53:37.898659 1071 raft_consensus.cc:385] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.899008 1071 raft_consensus.cc:740] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:37.899889 1071 consensus_queue.cc:260] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.901029 3039 tablet_loader.cc:96] loaded metadata for tablet 5a0f23d5212a46cc8f731d9c08019858 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1])
I20260810 07:53:37.902076 1064 raft_consensus.cc:359] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.902501 3039 tablet_loader.cc:96] loaded metadata for tablet 5a892c4bc5694118b514cc1baa56601d (table default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6])
I20260810 07:53:37.902896 1064 raft_consensus.cc:385] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:37.903209 1064 raft_consensus.cc:740] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:37.903396 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 64052.7us
I20260810 07:53:37.904018 1064 consensus_queue.cc:260] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.906441 1064 ts_tablet_manager.cc:1434] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260810 07:53:37.906786 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:37.907501 671 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:37.905598 3039 tablet_loader.cc:96] loaded metadata for tablet 5c1ddea2d9f3462092fcbaee7774cd01 (table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55])
I20260810 07:53:37.911466 3039 tablet_loader.cc:96] loaded metadata for tablet 5d7024a39aa040c2b00196f668b776e8 (table default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4])
I20260810 07:53:37.912830 3039 tablet_loader.cc:96] loaded metadata for tablet 5dad896bfa0e4c5cbc4907a75082b064 (table default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e])
I20260810 07:53:37.915022 3039 tablet_loader.cc:96] loaded metadata for tablet 5eb896c8922e412c97e2cc4891298830 (table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19])
I20260810 07:53:37.916100 3039 tablet_loader.cc:96] loaded metadata for tablet 613d1c158c7c4f2f8626536292d00fd1 (table default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133])
I20260810 07:53:37.917948 1047 tablet_bootstrap.cc:654] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:37.930542 4217 consensus_queue.cc:1048] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:37.935845 3039 tablet_loader.cc:96] loaded metadata for tablet 63812f4779914930b4e22d020a51a33b (table default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9])
I20260810 07:53:37.937129 3039 tablet_loader.cc:96] loaded metadata for tablet 666f36dafa92421cae47c94593ed8899 (table default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f])
I20260810 07:53:37.939204 3039 tablet_loader.cc:96] loaded metadata for tablet 69f1405e9526433a8fd727ed237f8c84 (table default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23])
I20260810 07:53:37.952986 4214 raft_consensus.cc:493] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.942549 3039 tablet_loader.cc:96] loaded metadata for tablet 6cd9484084294acfba9613fa49f369b2 (table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643])
I20260810 07:53:37.961678 3039 tablet_loader.cc:96] loaded metadata for tablet 6d67fef331994ca48f1dfce1e3b039f4 (table default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9])
I20260810 07:53:37.967062 3039 tablet_loader.cc:96] loaded metadata for tablet 6db16797cabc48fe8ec82355ce3f16be (table default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250])
I20260810 07:53:37.954054 4214 raft_consensus.cc:515] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:37.969008 3039 tablet_loader.cc:96] loaded metadata for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7 (table default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e])
I20260810 07:53:37.970914 3039 tablet_loader.cc:96] loaded metadata for tablet 705fc015928140a58db979445068365f (table default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219])
I20260810 07:53:37.972067 3039 tablet_loader.cc:96] loaded metadata for tablet 741404076635419bbbd6ce075fc44ccb (table default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264])
I20260810 07:53:37.974169 4189 raft_consensus.cc:493] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:37.975011 4189 raft_consensus.cc:515] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.975544 4219 raft_consensus.cc:493] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:37.976133 4219 raft_consensus.cc:515] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:37.978829 4189 leader_election.cc:290] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:37.979454 4219 leader_election.cc:290] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:37.980577 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 51249.7us
I20260810 07:53:37.984277 3039 tablet_loader.cc:96] loaded metadata for tablet 74d80c441b2b418994b46250a5c7af44 (table default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24])
I20260810 07:53:37.990527 3039 tablet_loader.cc:96] loaded metadata for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70])
I20260810 07:53:37.991845 3039 tablet_loader.cc:96] loaded metadata for tablet 7c075fdf1055477db9f2d56324f12a8a (table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7])
I20260810 07:53:38.004242 3039 tablet_loader.cc:96] loaded metadata for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017])
I20260810 07:53:38.006731 3039 tablet_loader.cc:96] loaded metadata for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067])
I20260810 07:53:38.006944 4214 leader_election.cc:290] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.008112 3039 tablet_loader.cc:96] loaded metadata for tablet 834a48ebb87a4b999722e05af1feab21 (table default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83])
W20260810 07:53:38.008339 32563 leader_election.cc:336] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845): Timed out: RequestConsensusVote RPC to 127.31.139.125:46845 timed out after 0.537s (SENT)
I20260810 07:53:38.008781 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
W20260810 07:53:37.963124 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 75723.4us
I20260810 07:53:38.017138 309 leader_election.cc:304] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19; no voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:38.013448 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19"
I20260810 07:53:38.017951 417 raft_consensus.cc:3060] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:38.025080 4004 raft_consensus.cc:2749] T ed74e441397344e09e924fb724ad55d6 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:38.034459 3039 tablet_loader.cc:96] loaded metadata for tablet 83b68614cd8444a08c7820a33c59442d (table default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a])
I20260810 07:53:38.037562 3039 tablet_loader.cc:96] loaded metadata for tablet 849197942f134542a387a10167fc7481 (table default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4])
I20260810 07:53:38.039788 3039 tablet_loader.cc:96] loaded metadata for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5])
I20260810 07:53:38.050014 1071 ts_tablet_manager.cc:1434] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.346s user 0.002s sys 0.005s
I20260810 07:53:38.049926 4226 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:38.050561 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 46679.6us
I20260810 07:53:38.051442 3039 tablet_loader.cc:96] loaded metadata for tablet 87ae23cb445e43fc9dc2f45097190fec (table default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de])
I20260810 07:53:38.052888 3039 tablet_loader.cc:96] loaded metadata for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f])
I20260810 07:53:38.061771 3039 tablet_loader.cc:96] loaded metadata for tablet 8c02aece0af740828f4009706445e60c (table default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855])
I20260810 07:53:38.073441 3039 tablet_loader.cc:96] loaded metadata for tablet 8ea09585009e4a3ea33f620182bfee60 (table default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27])
I20260810 07:53:38.074708 3039 tablet_loader.cc:96] loaded metadata for tablet 8ee78955a68a451f81e01a62a57aa4a4 (table default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7])
I20260810 07:53:38.075929 3039 tablet_loader.cc:96] loaded metadata for tablet 91c2e1bcd13a4c22b2141949b2d73188 (table default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f])
I20260810 07:53:38.080663 3039 tablet_loader.cc:96] loaded metadata for tablet 92c95cd95c034d9a9205221d07ae8615 (table default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0])
I20260810 07:53:38.089571 3039 tablet_loader.cc:96] loaded metadata for tablet 9350b151227d45e6bae8adddd8fff86b (table default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957])
I20260810 07:53:38.090934 3039 tablet_loader.cc:96] loaded metadata for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1])
I20260810 07:53:38.092265 3039 tablet_loader.cc:96] loaded metadata for tablet 93f62494bb034cd0b4f34f61652569c6 (table default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e])
I20260810 07:53:38.104373 3039 tablet_loader.cc:96] loaded metadata for tablet 94ced893d6804aa395b73f1f5ed085f9 (table default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca])
I20260810 07:53:38.106316 3039 tablet_loader.cc:96] loaded metadata for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72])
I20260810 07:53:38.107503 3039 tablet_loader.cc:96] loaded metadata for tablet 999f3e79ee4b4c368b60b202735c2556 (table default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6])
I20260810 07:53:38.108019 308 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:38.109194 4004 raft_consensus.cc:2804] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:53:38.109797 4004 raft_consensus.cc:493] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:38.110635 4004 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:38.116906 3039 tablet_loader.cc:96] loaded metadata for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661])
I20260810 07:53:38.119992 3039 tablet_loader.cc:96] loaded metadata for tablet 9f6e61aa223048f8bdc9eee07fa2066c (table default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812])
I20260810 07:53:38.122505 3039 tablet_loader.cc:96] loaded metadata for tablet a3642f29b38241be9db3cf37ccde2039 (table default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1])
I20260810 07:53:38.124433 3039 tablet_loader.cc:96] loaded metadata for tablet a4d1c8d270f744ee9849475c9128414b (table default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614])
I20260810 07:53:38.132915 3039 tablet_loader.cc:96] loaded metadata for tablet a57324aac10d42d191f72e2c2343d4ea (table default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a])
I20260810 07:53:38.136914 3039 tablet_loader.cc:96] loaded metadata for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426])
I20260810 07:53:38.056502 792 leader_election.cc:304] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters:
I20260810 07:53:38.138144 3039 tablet_loader.cc:96] loaded metadata for tablet a61dbafb4f7444e690cea0d003d38f22 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d])
I20260810 07:53:38.139245 3039 tablet_loader.cc:96] loaded metadata for tablet a7e077b6b69044c2b6d3d2bb7d99ee29 (table default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793])
I20260810 07:53:38.140327 3039 tablet_loader.cc:96] loaded metadata for tablet a86401e669fa4358ae33ce2249b3a98e (table default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45])
I20260810 07:53:38.142850 3039 tablet_loader.cc:96] loaded metadata for tablet a9f5a4321a504ed59e0e2ba21ac801f7 (table default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168])
I20260810 07:53:38.145444 1047 tablet_bootstrap.cc:492] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:38.145882 1047 ts_tablet_manager.cc:1403] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.258s user 0.035s sys 0.042s
I20260810 07:53:38.148314 1047 raft_consensus.cc:359] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.148996 1047 raft_consensus.cc:385] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.149281 1047 raft_consensus.cc:740] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.150015 1047 consensus_queue.cc:260] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.152047 1047 ts_tablet_manager.cc:1434] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:53:38.153926 3039 tablet_loader.cc:96] loaded metadata for tablet ab2e7619a25b437c8ee2c817255ca638 (table default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b])
I20260810 07:53:38.156558 3039 tablet_loader.cc:96] loaded metadata for tablet aed14fa26b9a46a4b3e9a43bac05a795 (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c])
I20260810 07:53:38.157832 3039 tablet_loader.cc:96] loaded metadata for tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e])
I20260810 07:53:38.161444 3039 tablet_loader.cc:96] loaded metadata for tablet b2cc9ff5b3344dc39e2e560d63f0ce92 (table default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf])
I20260810 07:53:38.162577 3039 tablet_loader.cc:96] loaded metadata for tablet b2fb91a223154dde8ecc2bebf2c0fff9 (table default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408])
I20260810 07:53:38.163692 3039 tablet_loader.cc:96] loaded metadata for tablet b305a232550e4d4babb19093ea1163f0 (table default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d])
I20260810 07:53:38.164845 3039 tablet_loader.cc:96] loaded metadata for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9])
I20260810 07:53:38.165930 3039 tablet_loader.cc:96] loaded metadata for tablet b6cb01ce3d97454d9d1e7735e51a76c4 (table default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97])
I20260810 07:53:38.167098 3039 tablet_loader.cc:96] loaded metadata for tablet b705e1630f124370af386436f325004b (table default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257])
I20260810 07:53:38.168485 3039 tablet_loader.cc:96] loaded metadata for tablet b7bbccbecf884768b3b171d9bd901b9b (table default.Table_b3adcec668f7473687d358c9b80c10f4 [id=b23efee8dd99419d9308ef2d5a907409])
I20260810 07:53:38.169595 3039 tablet_loader.cc:96] loaded metadata for tablet bdc0039db22d4812a1412e10175d2237 (table default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5])
I20260810 07:53:38.170653 3039 tablet_loader.cc:96] loaded metadata for tablet bf6663cbe7fd43a79265d87c2cc11a62 (table default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5])
I20260810 07:53:38.171705 3039 tablet_loader.cc:96] loaded metadata for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da])
I20260810 07:53:38.175565 3039 tablet_loader.cc:96] loaded metadata for tablet c47792551bb243b7b3f6de904c17c47b (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a])
I20260810 07:53:38.176653 3039 tablet_loader.cc:96] loaded metadata for tablet c6080c8b12014fe09d48e307d394f9c9 (table default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4])
W20260810 07:53:38.177176 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 120770us
I20260810 07:53:38.179327 3039 tablet_loader.cc:96] loaded metadata for tablet cc9959255b4d46e29b9164027638e38e (table default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866])
I20260810 07:53:38.180452 3039 tablet_loader.cc:96] loaded metadata for tablet cca378598d0c45178e497fd212638d77 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995])
I20260810 07:53:38.181173 4214 raft_consensus.cc:2804] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:38.181643 4214 raft_consensus.cc:493] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:38.182189 4214 raft_consensus.cc:3060] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:38.184067 3039 tablet_loader.cc:96] loaded metadata for tablet d3077b3fe04e406a89560229b76d8242 (table default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f])
I20260810 07:53:38.188797 3039 tablet_loader.cc:96] loaded metadata for tablet dd79131856f54395a3c5fb551961d4f7 (table default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d])
I20260810 07:53:38.189994 3039 tablet_loader.cc:96] loaded metadata for tablet e0a74b22a6864180956e763d2b6b6de6 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92])
I20260810 07:53:38.193138 3039 tablet_loader.cc:96] loaded metadata for tablet e20d46d62cdc48669d7474fae00660d7 (table default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446])
I20260810 07:53:38.194584 3039 tablet_loader.cc:96] loaded metadata for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8])
I20260810 07:53:38.196436 3039 tablet_loader.cc:96] loaded metadata for tablet e38ab80cd5c441f3ba20bf00577b223c (table default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18])
I20260810 07:53:38.200040 3039 tablet_loader.cc:96] loaded metadata for tablet e7529f4ac9fb457ba1b3c3bc90eb275d (table default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898])
I20260810 07:53:38.201972 3039 tablet_loader.cc:96] loaded metadata for tablet ebfc73a83a0f4048ad9dded62d074e19 (table default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43])
I20260810 07:53:38.100317 309 leader_election.cc:304] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:38.203116 3039 tablet_loader.cc:96] loaded metadata for tablet ed74e441397344e09e924fb724ad55d6 (table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37])
I20260810 07:53:38.203536 4096 raft_consensus.cc:2804] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:38.203944 4096 raft_consensus.cc:493] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:38.204334 4096 raft_consensus.cc:3060] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:38.205047 3039 tablet_loader.cc:96] loaded metadata for tablet f0606ccac5d04aa89d804cf9168018b8 (table default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4])
I20260810 07:53:38.207762 3039 tablet_loader.cc:96] loaded metadata for tablet f4a46a037cae4f599398b289bb6a2d84 (table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c])
I20260810 07:53:38.209080 3039 tablet_loader.cc:96] loaded metadata for tablet f65480f06a3c4b2886aa2485f81e16f2 (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b])
I20260810 07:53:38.210450 3039 tablet_loader.cc:96] loaded metadata for tablet f6b54db926fa4032b6408d3d309de35b (table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017])
W20260810 07:53:38.210973 308 consensus_peers.cc:597] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 -> Peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Couldn't send request to peer b5bfa873accc428ab48ac19e1207f88d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:38.211552 3039 tablet_loader.cc:96] loaded metadata for tablet f6cd26a04d3c417d87091f05bc5cb5cd (table default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853])
W20260810 07:53:38.209177 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 118215us
I20260810 07:53:38.212713 3039 tablet_loader.cc:96] loaded metadata for tablet f8415b6376a740388487f7ae4aebea52 (table default.Table_b041641ad99146e8ac57b26f842b8976 [id=0ab295f37b5e482482053f67fe0719a6])
I20260810 07:53:38.212734 308 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:38.213501 3997 raft_consensus.cc:2804] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:53:38.213930 3997 raft_consensus.cc:493] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:38.214361 3997 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:38.214781 3039 tablet_loader.cc:96] loaded metadata for tablet fc12205a097c496cba5b777b98f27914 (table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73])
I20260810 07:53:38.225327 3997 raft_consensus.cc:515] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:38.208448 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 108226us
I20260810 07:53:38.248864 4153 consensus_queue.cc:1048] T a6077111ffb2411892b62d806ea3c85b P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:38.246737 1071 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
W20260810 07:53:38.246090 3039 catalog_manager.cc:1512] Time spent T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Loading table and tablet metadata into memory: real 17.215s user 0.484s sys 0.081s
I20260810 07:53:38.257071 3039 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:53:38.257296 32486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.258487 32486 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 4) took 16046 ms (client timeout 548 ms). Trace:
W20260810 07:53:38.258900 32486 rpcz_store.cc:269] 0810 07:53:22.211900 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:22.212690 (+ 790us) service_pool.cc:224] Handling call
0810 07:53:38.258389 (+16045699us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.260102 32486 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 33) took 11338 ms (client timeout 581 ms). Trace:
I20260810 07:53:38.260995 32484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.261725 312 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.139.124:36273, user_credentials={real_user=slave}} blocked reactor thread for 91503.2us
W20260810 07:53:38.261869 32484 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 27) took 16131 ms (client timeout 544 ms). Trace:
W20260810 07:53:38.262188 32484 rpcz_store.cc:269] 0810 07:53:22.130267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:22.130406 (+ 139us) service_pool.cc:224] Handling call
0810 07:53:38.261793 (+16131387us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.263394 32484 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 9) took 10962 ms (client timeout 680 ms). Trace:
W20260810 07:53:38.263753 32484 rpcz_store.cc:269] 0810 07:53:27.301089 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.263140 (+10962051us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.263293 (+ 153us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:38.265033 32480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.265929 32480 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 5) took 15450 ms (client timeout 548 ms). Trace:
W20260810 07:53:38.266331 32480 rpcz_store.cc:269] 0810 07:53:22.815806 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:22.816065 (+ 259us) service_pool.cc:224] Handling call
0810 07:53:38.265854 (+15449789us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.267103 32480 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 35) took 9344 ms (client timeout 654 ms). Trace:
W20260810 07:53:38.267388 32480 rpcz_store.cc:269] 0810 07:53:28.922368 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.266954 (+9344586us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.267059 (+ 105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.268092 32480 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 10) took 8709 ms (client timeout 591 ms). Trace:
W20260810 07:53:38.268379 32480 rpcz_store.cc:269] 0810 07:53:29.559185 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.267948 (+8708763us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.268048 (+ 100us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:38.269011 32474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.276320 32474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 32) took 12817 ms (client timeout 514 ms). Trace:
W20260810 07:53:38.276700 32474 rpcz_store.cc:269] 0810 07:53:25.459330 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:25.459532 (+ 202us) service_pool.cc:224] Handling call
0810 07:53:38.276227 (+12816695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:38.269668 32478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:38.270309 32479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.278406 32479 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 6) took 13506 ms (client timeout 566 ms). Trace:
W20260810 07:53:38.278723 32479 rpcz_store.cc:269] 0810 07:53:24.772198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:24.772435 (+ 237us) service_pool.cc:224] Handling call
0810 07:53:38.278328 (+13505893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:38.279091 886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:38.279595 32479 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 11) took 8031 ms (client timeout 591 ms). Trace:
I20260810 07:53:38.279697 886 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:38.279911 32479 rpcz_store.cc:269] 0810 07:53:30.247926 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.279432 (+8031506us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.279545 (+ 113us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.281098 32478 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 31) took 13421 ms (client timeout 514 ms). Trace:
W20260810 07:53:38.281546 32478 rpcz_store.cc:269] 0810 07:53:24.859703 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:24.867258 (+ 7555us) service_pool.cc:224] Handling call
0810 07:53:38.280992 (+13413734us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.282845 32478 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 37) took 6148 ms (client timeout 537 ms). Trace:
W20260810 07:53:38.283304 32478 rpcz_store.cc:269] 0810 07:53:32.134868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.282601 (+6147733us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.282770 (+ 169us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.284113 32478 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 13) took 5785 ms (client timeout 717 ms). Trace:
I20260810 07:53:38.272535 32461 rpcz_store.cc:275] Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:43806 (request call id 376) took 16758 ms. Trace:
I20260810 07:53:38.284814 32461 rpcz_store.cc:276] 0810 07:53:21.514211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:21.514905 (+ 694us) service_pool.cc:224] Handling call
0810 07:53:38.272183 (+16757278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:38.285172 3039 catalog_manager.cc:1295] Loaded cluster ID: 35e92927efed4ea9952bfdaa3b28c875
I20260810 07:53:38.274322 32483 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54958 (request call id 29) took 15987 ms. Trace:
I20260810 07:53:38.285451 3039 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:53:38.285605 32483 rpcz_store.cc:276] 0810 07:53:22.286582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:22.286953 (+ 371us) service_pool.cc:224] Handling call
0810 07:53:38.273954 (+15987001us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.287648 32483 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 38) took 5551 ms (client timeout 537 ms). Trace:
W20260810 07:53:38.287932 32483 rpcz_store.cc:269] 0810 07:53:32.736642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.287489 (+5550847us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.287597 (+ 108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.288635 32483 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 39) took 3705 ms (client timeout 643 ms). Trace:
W20260810 07:53:38.288944 32483 rpcz_store.cc:269] 0810 07:53:34.583637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.288491 (+3704854us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.288570 (+ 79us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.289739 32483 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 14) took 3453 ms (client timeout 745 ms). Trace:
W20260810 07:53:38.290024 32483 rpcz_store.cc:269] 0810 07:53:34.836323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.289595 (+3453272us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.289689 (+ 94us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.290720 32483 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 40) took 2949 ms (client timeout 643 ms). Trace:
W20260810 07:53:38.291003 32483 rpcz_store.cc:269] 0810 07:53:35.341128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.290589 (+2949461us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.290671 (+ 82us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.291793 32483 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 15) took 2616 ms (client timeout 745 ms). Trace:
W20260810 07:53:38.292097 32483 rpcz_store.cc:269] 0810 07:53:35.675089 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.291648 (+2616559us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.291741 (+ 93us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:38.271214 32471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.293460 32471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 8) took 11689 ms (client timeout 680 ms). Trace:
W20260810 07:53:38.293913 32471 rpcz_store.cc:269] 0810 07:53:26.604076 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:26.604323 (+ 247us) service_pool.cc:224] Handling call
0810 07:53:38.293355 (+11689032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:38.257073 32482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.295056 32482 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 30) took 15604 ms (client timeout 544 ms). Trace:
W20260810 07:53:38.295370 32482 rpcz_store.cc:269] 0810 07:53:22.690643 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:22.690876 (+ 233us) service_pool.cc:224] Handling call
0810 07:53:38.294986 (+15604110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:53:38.256542 32473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.296443 32473 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 7) took 12814 ms (client timeout 566 ms). Trace:
W20260810 07:53:38.296808 32473 rpcz_store.cc:269] 0810 07:53:25.481711 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:25.481924 (+ 213us) service_pool.cc:224] Handling call
0810 07:53:38.296364 (+12814440us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260810 07:53:38.280704 32479 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 12) took 6660 ms (client timeout 717 ms). Trace:
W20260810 07:53:38.297981 32479 rpcz_store.cc:269] 0810 07:53:31.620558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.280532 (+6659974us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.280653 (+ 121us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.260448 32486 rpcz_store.cc:269] 0810 07:53:26.921904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.259829 (+11337925us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.260025 (+ 196us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.284430 32478 rpcz_store.cc:269] 0810 07:53:32.499187 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.283965 (+5784778us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.284066 (+ 101us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.303580 32471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 41) took 1418 ms (client timeout 537 ms). Trace:
W20260810 07:53:38.304077 32471 rpcz_store.cc:269] 0810 07:53:36.885099 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.303344 (+1418245us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.303505 (+ 161us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.305706 32482 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60564 (request call id 16) took 1097 ms (client timeout 689 ms). Trace:
W20260810 07:53:38.306008 32482 rpcz_store.cc:269] 0810 07:53:37.208555 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.305526 (+1096971us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.305652 (+ 126us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.306663 32482 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 42) took 835 ms (client timeout 537 ms). Trace:
W20260810 07:53:38.306949 32482 rpcz_store.cc:269] 0810 07:53:37.471294 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.306535 (+835241us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.306616 (+ 81us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:38.308104 32482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 406 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:38.308791 32474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 36) took 8374 ms (client timeout 654 ms). Trace:
W20260810 07:53:38.309229 32474 rpcz_store.cc:269] 0810 07:53:29.902918 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.277457 (+8374539us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.277598 (+ 141us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260810 07:53:38.264598 32484 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54958 (request call id 34) took 10724 ms (client timeout 581 ms). Trace:
W20260810 07:53:38.311008 32484 rpcz_store.cc:269] 0810 07:53:27.540341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.264438 (+10724097us) service_pool.cc:211] Skipping call since client already timed out
0810 07:53:38.264550 (+ 112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260810 07:53:38.312809 3039 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:53:38.316419 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.289917 4233 consensus_queue.cc:1048] T f6cd26a04d3c417d87091f05bc5cb5cd P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:38.339123 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:38.343777 3039 catalog_manager.cc:6103] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Loaded TSK: 0
I20260810 07:53:38.345057 3039 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:53:38.345537 4206 raft_consensus.cc:493] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.346606 4206 raft_consensus.cc:515] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.334687 4153 raft_consensus.cc:493] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.351301 4153 raft_consensus.cc:515] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.364142 1064 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:38.370824 1064 tablet_bootstrap.cc:654] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.371362 4206 leader_election.cc:290] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.312381 417 raft_consensus.cc:2468] T bdc0039db22d4812a1412e10175d2237 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
W20260810 07:53:38.379658 573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 73812.5us
I20260810 07:53:38.361222 3997 leader_election.cc:290] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.387584 4244 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:38.388837 2560 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.125:46845, user_credentials={real_user=slave}} blocked reactor thread for 49806.1us
I20260810 07:53:38.374591 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:38.389612 892 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:38.390080 4153 leader_election.cc:290] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.406064 573 leader_election.cc:304] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:38.406751 4212 raft_consensus.cc:2804] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:38.407220 4212 raft_consensus.cc:697] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:38.408033 4212 consensus_queue.cc:237] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.411242 1064 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:38.411775 1064 ts_tablet_manager.cc:1403] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.001s
I20260810 07:53:38.414490 1064 raft_consensus.cc:359] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.415189 1064 raft_consensus.cc:385] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.415494 1064 raft_consensus.cc:740] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.416343 1064 consensus_queue.cc:260] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.418707 1064 ts_tablet_manager.cc:1434] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:53:38.431653 1064 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:38.433970 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 79947.6us
I20260810 07:53:38.437338 1064 tablet_bootstrap.cc:654] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.445722 32461 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b3adcec668f7473687d358c9b80c10f4" } new_table_name: "default.Table_110c350dc5474f2fbe9bbd9410aac109" modify_external_catalogs: true
I20260810 07:53:38.446614 32463 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_b041641ad99146e8ac57b26f842b8976" } new_table_name: "default.Table_fe0f90fb5d4445a0abb442761bb12227" modify_external_catalogs: true
I20260810 07:53:38.449858 32470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_1315259f9b4e4425a2f99c5125e402bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:53:38.449383 32467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_10ebf20223374508adbac3c536614ad3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:38.451696 32470 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1315259f9b4e4425a2f99c5125e402bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:53:38.451956 32467 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10ebf20223374508adbac3c536614ad3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:38.457073 32468 master_service.cc:960] ReplaceTablet: received request to replace tablet 705fc015928140a58db979445068365f from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:38.464468 32460 master_service.cc:960] ReplaceTablet: received request to replace tablet 5c1ddea2d9f3462092fcbaee7774cd01 from {username='slave'} at 127.0.0.1:43806
I20260810 07:53:38.464584 32469 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_52b389643dbe4064ba2010a1d2d6c8aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:38.466687 32469 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52b389643dbe4064ba2010a1d2d6c8aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:38.468930 32465 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "default.Table_7b1b0b8833e640e1bdddb0a030106d84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:53:38.470573 32465 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b1b0b8833e640e1bdddb0a030106d84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:53:38.475430 32464 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_1af94d6703df4a2eb7e30cdf6301bdce" } modify_external_catalogs: true
I20260810 07:53:38.475809 32462 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_20e00f06bba84d53bf31edd76a1a3940" } modify_external_catalogs: true
I20260810 07:53:38.476202 32464 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_1af94d6703df4a2eb7e30cdf6301bdce" } modify_external_catalogs: true
I20260810 07:53:38.476613 32462 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43806:
table { table_name: "default.Table_20e00f06bba84d53bf31edd76a1a3940" } modify_external_catalogs: true
I20260810 07:53:38.500216 1047 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:38.506093 1064 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:38.506498 1064 ts_tablet_manager.cc:1403] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.000s
I20260810 07:53:38.508880 1064 raft_consensus.cc:359] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.509546 1064 raft_consensus.cc:385] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.509869 1064 raft_consensus.cc:740] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.510612 1064 consensus_queue.cc:260] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.515185 1064 ts_tablet_manager.cc:1434] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260810 07:53:38.509375 4214 raft_consensus.cc:515] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.521500 1047 tablet_bootstrap.cc:654] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.525424 4214 leader_election.cc:290] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.533352 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.535054 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.536512 1047 tablet_bootstrap.cc:492] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:38.537030 1047 ts_tablet_manager.cc:1403] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.037s user 0.014s sys 0.003s
I20260810 07:53:38.538642 1071 tablet_bootstrap.cc:654] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.540246 1047 raft_consensus.cc:359] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.541188 1047 raft_consensus.cc:385] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.541633 1047 raft_consensus.cc:740] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.542541 1047 consensus_queue.cc:260] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.546005 1047 ts_tablet_manager.cc:1434] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260810 07:53:38.547060 1047 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:38.547833 4096 raft_consensus.cc:515] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:38.553699 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 117471us
I20260810 07:53:38.554220 1047 tablet_bootstrap.cc:654] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.565728 1064 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:38.573956 1064 tablet_bootstrap.cc:654] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.580853 677 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:38.592561 4214 raft_consensus.cc:493] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.593139 4214 raft_consensus.cc:515] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.595186 4214 leader_election.cc:290] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:38.595736 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:38.596446 671 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:38.597397 4214 raft_consensus.cc:493] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.597852 4214 raft_consensus.cc:515] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.600059 4214 leader_election.cc:290] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:38.613718 4214 raft_consensus.cc:493] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.614430 4214 raft_consensus.cc:515] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.616140 4206 raft_consensus.cc:493] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.616880 4206 raft_consensus.cc:515] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.617827 671 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
I20260810 07:53:38.619566 4206 leader_election.cc:290] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:38.622202 4214 leader_election.cc:290] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:38.623020 1047 tablet_bootstrap.cc:492] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:38.623699 1047 ts_tablet_manager.cc:1403] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.077s user 0.013s sys 0.002s
W20260810 07:53:38.624857 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 70466.4us
I20260810 07:53:38.634807 4042 raft_consensus.cc:493] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.635473 4042 raft_consensus.cc:515] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.634059 1047 raft_consensus.cc:359] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.638815 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.639451 1047 raft_consensus.cc:385] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.640010 670 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 2: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:38.640005 1047 raft_consensus.cc:740] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.638837 4042 leader_election.cc:290] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.641258 1047 consensus_queue.cc:260] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.642620 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.643252 677 raft_consensus.cc:2468] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:38.645649 1047 ts_tablet_manager.cc:1434] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
I20260810 07:53:38.646929 1047 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:38.641970 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.648586 671 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:38.401628 4004 raft_consensus.cc:515] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.667198 1047 tablet_bootstrap.cc:654] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.677975 4238 raft_consensus.cc:493] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.678776 4238 raft_consensus.cc:515] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.681399 4238 leader_election.cc:290] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.684693 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.685389 671 raft_consensus.cc:2468] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:38.693611 4004 leader_election.cc:290] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.706647 4096 leader_election.cc:290] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:38.735999 4206 raft_consensus.cc:493] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.736527 4206 raft_consensus.cc:515] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.739359 4206 leader_election.cc:290] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:38.754197 1071 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:38.755892 1064 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:38.756989 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.757850 403 raft_consensus.cc:2393] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:38.758381 412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.758759 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.759063 412 raft_consensus.cc:2393] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 2: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:38.759419 403 raft_consensus.cc:2393] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 2: Already voted for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in this term.
I20260810 07:53:38.760234 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.760828 417 raft_consensus.cc:2468] T 05ef1be8eaa84e6bb5586e68268799a0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:38.781754 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:38.782497 403 raft_consensus.cc:2468] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:38.789165 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:38.791090 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8f390b0db64e1d96c3d8686274cbbf" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:38.791666 876 raft_consensus.cc:3060] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:38.756484 1071 ts_tablet_manager.cc:1403] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.510s user 0.006s sys 0.006s
I20260810 07:53:38.761696 1064 ts_tablet_manager.cc:1403] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.196s user 0.014s sys 0.000s
I20260810 07:53:38.802443 876 raft_consensus.cc:2468] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
I20260810 07:53:38.816896 4206 raft_consensus.cc:493] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.817356 4206 raft_consensus.cc:515] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.820930 4206 leader_election.cc:290] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.801789 1064 raft_consensus.cc:359] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.826004 1064 raft_consensus.cc:385] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.826316 1064 raft_consensus.cc:740] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.827107 1064 consensus_queue.cc:260] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.828836 1064 ts_tablet_manager.cc:1434] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260810 07:53:38.865736 1071 raft_consensus.cc:359] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.866397 1071 raft_consensus.cc:385] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:38.867915 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:38.869356 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.868633 665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ff256b314e479d981333d9cbfc1656" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:38.871628 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:38.872196 666 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:38.829772 1064 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:38.883826 671 raft_consensus.cc:3060] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:38.890419 1071 raft_consensus.cc:740] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:38.892000 1071 consensus_queue.cc:260] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.896572 1064 tablet_bootstrap.cc:654] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:38.901600 1071 ts_tablet_manager.cc:1434] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.104s user 0.006s sys 0.000s
W20260810 07:53:38.905380 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 59714.6us
I20260810 07:53:38.908449 671 raft_consensus.cc:2468] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
W20260810 07:53:38.912487 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 146621us
I20260810 07:53:38.931027 308 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:38.941763 4242 raft_consensus.cc:2804] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:53:38.942363 4242 raft_consensus.cc:697] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:38.943405 4242 consensus_queue.cc:237] T 2a8f390b0db64e1d96c3d8686274cbbf P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.959874 308 leader_election.cc:304] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:38.962888 4246 raft_consensus.cc:2804] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:53:38.963449 4246 raft_consensus.cc:493] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:38.963964 4246 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Advancing to term 2
W20260810 07:53:38.981344 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 46465.7us
I20260810 07:53:38.982710 4206 raft_consensus.cc:493] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:38.983223 4206 raft_consensus.cc:515] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:38.984175 4214 raft_consensus.cc:493] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:38.984647 4214 raft_consensus.cc:515] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:38.985569 4206 leader_election.cc:290] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.986683 4214 leader_election.cc:290] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:38.999172 1047 tablet_bootstrap.cc:492] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:38.999711 1047 ts_tablet_manager.cc:1403] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.353s user 0.039s sys 0.040s
I20260810 07:53:39.001281 666 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
W20260810 07:53:39.007306 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 44669.9us
I20260810 07:53:39.012960 4206 raft_consensus.cc:493] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:39.013463 4206 raft_consensus.cc:515] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.015609 4206 leader_election.cc:290] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:39.013650 4214 raft_consensus.cc:493] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
W20260810 07:53:39.015733 792 leader_election.cc:343] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:39.016180 4214 raft_consensus.cc:515] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.016947 4206 raft_consensus.cc:493] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b5bfa873accc428ab48ac19e1207f88d)
I20260810 07:53:39.017397 4206 raft_consensus.cc:515] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:39.019309 4206 leader_election.cc:290] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:39.020098 4214 leader_election.cc:290] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:39.028802 573 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s user 0.012s sys 0.005s
W20260810 07:53:39.029163 573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s user 0.012s sys 0.005s
I20260810 07:53:39.030877 4214 consensus_queue.cc:1048] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:53:39.032138 308 leader_election.cc:343] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:39.033519 792 leader_election.cc:343] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:39.038589 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705fc015928140a58db979445068365f" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:39.053336 889 raft_consensus.cc:3060] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:39.054323 1064 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.054908 1064 ts_tablet_manager.cc:1403] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.225s user 0.018s sys 0.018s
I20260810 07:53:39.038944 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:39.057101 875 raft_consensus.cc:3060] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:39.039623 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ff256b314e479d981333d9cbfc1656" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.010902 1047 raft_consensus.cc:359] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.064384 1047 raft_consensus.cc:385] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.064764 1047 raft_consensus.cc:740] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.074226 1064 raft_consensus.cc:359] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.076167 889 raft_consensus.cc:2468] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 1.
I20260810 07:53:39.035878 1071 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:39.086212 1064 raft_consensus.cc:385] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.083170 411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:39.102382 1064 raft_consensus.cc:740] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.103165 1064 consensus_queue.cc:260] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.104259 876 raft_consensus.cc:2468] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:39.104921 1064 ts_tablet_manager.cc:1434] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.050s user 0.010s sys 0.001s
I20260810 07:53:39.106019 1064 tablet_bootstrap.cc:492] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:39.110160 1071 tablet_bootstrap.cc:654] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:39.112910 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 79393us
W20260810 07:53:39.118587 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 86483.6us
I20260810 07:53:39.119341 308 leader_election.cc:304] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.122764 1064 tablet_bootstrap.cc:654] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.123598 4246 raft_consensus.cc:515] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.065501 1047 consensus_queue.cc:260] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.136826 1064 tablet_bootstrap.cc:492] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.137324 1064 ts_tablet_manager.cc:1403] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.008s
I20260810 07:53:39.140698 1064 raft_consensus.cc:359] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.141603 1064 raft_consensus.cc:385] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.141932 1064 raft_consensus.cc:740] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.142930 1064 consensus_queue.cc:260] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.079034 875 raft_consensus.cc:2468] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
I20260810 07:53:39.150141 373 tablet_service.cc:1558] Processing DeleteTablet for tablet 371fa2534984499b95c0945af358e8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:52:59 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:39.153998 4246 leader_election.cc:290] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:39.161755 1047 ts_tablet_manager.cc:1434] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.162s user 0.006s sys 0.001s
I20260810 07:53:39.168386 4004 raft_consensus.cc:2804] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:39.169082 4004 raft_consensus.cc:697] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:39.169633 32468 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 705fc015928140a58db979445068365f
I20260810 07:53:39.170097 4004 consensus_queue.cc:237] T 705fc015928140a58db979445068365f P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.171425 32468 catalog_manager.cc:6807] ReplaceTablet: tablet 705fc015928140a58db979445068365f deleted and replaced by tablet fce424ac9b4048c4a988f0fa48acbf46
I20260810 07:53:39.175450 856 tablet_service.cc:1558] Processing DeleteTablet for tablet 705fc015928140a58db979445068365f with delete_type TABLET_DATA_DELETED (replaced by tablet fce424ac9b4048c4a988f0fa48acbf46) from {username='slave'} at 127.0.0.1:55008
W20260810 07:53:39.176479 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 57200.7us
I20260810 07:53:39.179790 308 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.180743 4242 raft_consensus.cc:2804] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:53:39.181269 4242 raft_consensus.cc:697] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:39.181811 630 tablet_service.cc:1558] Processing DeleteTablet for tablet 705fc015928140a58db979445068365f with delete_type TABLET_DATA_DELETED (replaced by tablet fce424ac9b4048c4a988f0fa48acbf46) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:39.182356 4242 consensus_queue.cc:237] T 1d25b4b974034eea9f1256fb4e2a503c P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:39.198534 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 56738.7us
I20260810 07:53:39.199787 4217 consensus_queue.cc:1048] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:39.200133 792 leader_election.cc:304] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.205789 1064 ts_tablet_manager.cc:1434] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.068s user 0.007s sys 0.000s
I20260810 07:53:39.207134 1064 tablet_bootstrap.cc:492] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:39.210595 380 tablet_service.cc:1558] Processing DeleteTablet for tablet 705fc015928140a58db979445068365f with delete_type TABLET_DATA_DELETED (replaced by tablet fce424ac9b4048c4a988f0fa48acbf46) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:39.216996 1047 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:39.221333 4214 raft_consensus.cc:2764] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 07:53:39.223479 792 leader_election.cc:304] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.227583 4305 tablet_replica.cc:333] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:39.263103 4305 raft_consensus.cc:2243] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:39.263788 4305 raft_consensus.cc:2272] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:39.228339 4304 tablet_replica.cc:333] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:39.244583 1064 tablet_bootstrap.cc:654] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.266530 4214 raft_consensus.cc:2764] T 1d25b4b974034eea9f1256fb4e2a503c P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260810 07:53:39.268054 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 44671.4us
I20260810 07:53:39.274760 32460 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 5c1ddea2d9f3462092fcbaee7774cd01
I20260810 07:53:39.276396 32460 catalog_manager.cc:6807] ReplaceTablet: tablet 5c1ddea2d9f3462092fcbaee7774cd01 deleted and replaced by tablet 9ae857bd2c124682bbd682d81dc591a5
I20260810 07:53:39.277530 856 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae857bd2c124682bbd682d81dc591a5) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:39.277794 630 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae857bd2c124682bbd682d81dc591a5) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:39.286213 4305 ts_tablet_manager.cc:1916] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:39.287688 1047 tablet_bootstrap.cc:654] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.297053 32301 master-stress-test.cc:581] Killed test thread 1/10
I20260810 07:53:39.297627 32301 master-stress-test.cc:579] Killing test thread 2/10
I20260810 07:53:39.297931 32301 master-stress-test.cc:581] Killed test thread 2/10
I20260810 07:53:39.298179 32301 master-stress-test.cc:579] Killing test thread 3/10
I20260810 07:53:39.299428 1064 tablet_bootstrap.cc:492] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.299890 1064 ts_tablet_manager.cc:1403] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.093s user 0.011s sys 0.003s
I20260810 07:53:39.302798 1064 raft_consensus.cc:359] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.303473 1064 raft_consensus.cc:385] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.303735 1064 raft_consensus.cc:740] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.309929 4219 consensus_queue.cc:1048] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:39.327386 32301 master-stress-test.cc:581] Killed test thread 3/10
I20260810 07:53:39.327713 32301 master-stress-test.cc:579] Killing test thread 4/10
I20260810 07:53:39.327987 32301 master-stress-test.cc:581] Killed test thread 4/10
I20260810 07:53:39.328215 32301 master-stress-test.cc:579] Killing test thread 5/10
I20260810 07:53:39.328487 32301 master-stress-test.cc:581] Killed test thread 5/10
I20260810 07:53:39.328734 32301 master-stress-test.cc:579] Killing test thread 6/10
I20260810 07:53:39.329327 32301 master-stress-test.cc:581] Killed test thread 6/10
I20260810 07:53:39.329578 32301 master-stress-test.cc:579] Killing test thread 7/10
I20260810 07:53:39.304406 1064 consensus_queue.cc:260] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:39.314883 792 leader_election.cc:343] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:39.341707 1064 ts_tablet_manager.cc:1434] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.042s user 0.003s sys 0.004s
I20260810 07:53:39.342806 1064 tablet_bootstrap.cc:492] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:39.344144 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.345052 875 raft_consensus.cc:2393] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:39.346678 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ee78955a68a451f81e01a62a57aa4a4" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:39.347368 875 raft_consensus.cc:3060] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:39.350426 1047 tablet_bootstrap.cc:492] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:39.350909 1047 ts_tablet_manager.cc:1403] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.134s user 0.007s sys 0.007s
I20260810 07:53:39.357673 1047 raft_consensus.cc:359] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.358369 1047 raft_consensus.cc:385] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.358651 1047 raft_consensus.cc:740] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.383829 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38ab80cd5c441f3ba20bf00577b223c" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.387602 4219 raft_consensus.cc:493] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.388322 4219 raft_consensus.cc:515] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.391278 4219 leader_election.cc:290] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:39.393349 875 raft_consensus.cc:2468] T 8ee78955a68a451f81e01a62a57aa4a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:39.395892 4313 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:39.396469 4313 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:39.399652 4206 raft_consensus.cc:493] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.400172 4206 raft_consensus.cc:515] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.402212 4206 leader_election.cc:290] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:39.408507 1064 tablet_bootstrap.cc:654] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.424093 32464 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet afbac73bbeb44af6b39ab23c5cf79eed
I20260810 07:53:39.424844 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:39.387269 1047 consensus_queue.cc:260] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.425390 666 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:39.427198 856 tablet_service.cc:1558] Processing DeleteTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:39.429282 1047 ts_tablet_manager.cc:1434] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.078s user 0.006s sys 0.001s
I20260810 07:53:39.435034 630 tablet_service.cc:1558] Processing DeleteTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:39.439119 666 raft_consensus.cc:2468] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 2.
I20260810 07:53:39.440052 1047 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
W20260810 07:53:39.447906 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 285298us
I20260810 07:53:39.461242 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:39.462271 875 raft_consensus.cc:2393] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:39.469151 1047 tablet_bootstrap.cc:654] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.486392 4304 raft_consensus.cc:2243] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:39.487020 4304 raft_consensus.cc:2272] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:39.492005 4304 ts_tablet_manager.cc:1916] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:39.499138 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 407 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:39.501297 4004 raft_consensus.cc:493] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:39.502863 4004 raft_consensus.cc:515] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.505287 4313 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:39.508317 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 407 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:39.488601 1064 tablet_bootstrap.cc:492] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.510663 1064 ts_tablet_manager.cc:1403] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.168s user 0.011s sys 0.003s
I20260810 07:53:39.511893 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:39.515393 4305 ts_tablet_manager.cc:1929] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:39.515986 4305 log.cc:1199] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/705fc015928140a58db979445068365f
I20260810 07:53:39.517154 4305 ts_tablet_manager.cc:1950] T 705fc015928140a58db979445068365f P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:39.533092 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.533399 1047 tablet_bootstrap.cc:492] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:39.535696 1047 ts_tablet_manager.cc:1403] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.096s user 0.009s sys 0.004s
I20260810 07:53:39.536736 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.537425 1064 raft_consensus.cc:359] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.538077 1064 raft_consensus.cc:385] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.538406 1064 raft_consensus.cc:740] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.538849 1047 raft_consensus.cc:359] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.539551 1047 raft_consensus.cc:385] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.549692 499 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:53:39.549762 4298 tablet_replica.cc:333] T 371fa2534984499b95c0945af358e8fc P 60781ef3c8b84a8788f43d77e1bb3b19: stopping tablet replica
I20260810 07:53:39.504525 4238 raft_consensus.cc:493] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.556746 1071 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:39.560097 1071 ts_tablet_manager.cc:1403] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.524s user 0.013s sys 0.017s
I20260810 07:53:39.539850 1047 raft_consensus.cc:740] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.563673 1047 consensus_queue.cc:260] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.565433 4004 leader_election.cc:290] T 1f8941c49c0a4fba9f7de2aabb790901 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:39.566134 4238 raft_consensus.cc:515] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.569679 1064 consensus_queue.cc:260] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.571028 1047 ts_tablet_manager.cc:1434] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260810 07:53:39.568107 4238 leader_election.cc:290] T 8c02aece0af740828f4009706445e60c P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:39.572811 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6077111ffb2411892b62d806ea3c85b" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
W20260810 07:53:39.580933 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 130633us
I20260810 07:53:39.586813 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.587518 892 raft_consensus.cc:2468] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:39.590420 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.591025 666 raft_consensus.cc:2468] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:39.596498 4206 raft_consensus.cc:493] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.597015 4206 raft_consensus.cc:515] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.598867 1064 ts_tablet_manager.cc:1434] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.088s user 0.005s sys 0.000s
I20260810 07:53:39.598878 4206 leader_election.cc:290] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:39.600029 1064 tablet_bootstrap.cc:492] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:39.605769 1047 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:39.606667 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc0039db22d4812a1412e10175d2237" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:39.607872 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 705fc015928140a58db979445068365f (table default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]) successfully deleted
I20260810 07:53:39.608161 4305 tablet_replica.cc:333] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:39.622347 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4863f3f03104d31ad4dafda6f33c9af" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.628456 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a95df8c34ad4ff4bfe70cea1e0ebb92" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.631685 1071 raft_consensus.cc:359] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.632432 1071 raft_consensus.cc:385] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.634963 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbccbecf884768b3b171d9bd901b9b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.636518 1071 raft_consensus.cc:740] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.640661 1071 consensus_queue.cc:260] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.641090 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afbac73bbeb44af6b39ab23c5cf79eed" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.644562 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6080c8b12014fe09d48e307d394f9c9" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:39.649551 309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 54550.1us
I20260810 07:53:39.649977 1047 tablet_bootstrap.cc:654] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.663616 4305 raft_consensus.cc:2243] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:39.642599 1071 ts_tablet_manager.cc:1434] T dd79131856f54395a3c5fb551961d4f7 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent starting tablet: real 0.082s user 0.000s sys 0.007s
I20260810 07:53:39.664609 4305 raft_consensus.cc:2272] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:39.667565 4305 ts_tablet_manager.cc:1916] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:39.681317 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2905f0597c07439989a00a213fc2194f" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.636603 4304 ts_tablet_manager.cc:1929] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:53:39.684085 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c95cd95c034d9a9205221d07ae8615" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.684700 4304 log.cc:1199] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/705fc015928140a58db979445068365f
I20260810 07:53:39.685796 4304 ts_tablet_manager.cc:1950] T 705fc015928140a58db979445068365f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:39.686676 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02aece0af740828f4009706445e60c" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.694336 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 705fc015928140a58db979445068365f (table default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]) successfully deleted
I20260810 07:53:39.703387 4304 tablet_replica.cc:333] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:39.710421 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c49060c69f2434db4e9f83f2147d7d1" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.711607 1047 tablet_bootstrap.cc:492] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:39.712065 1047 ts_tablet_manager.cc:1403] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.107s user 0.010s sys 0.001s
I20260810 07:53:39.714886 1047 raft_consensus.cc:359] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.715521 1047 raft_consensus.cc:385] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.715761 1047 raft_consensus.cc:740] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.717084 4304 raft_consensus.cc:2243] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:39.717680 4304 raft_consensus.cc:2272] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:39.717855 1064 tablet_bootstrap.cc:654] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.719647 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b54db926fa4032b6408d3d309de35b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.716378 1047 consensus_queue.cc:260] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.732348 4313 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:39.742800 4304 ts_tablet_manager.cc:1916] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:39.746970 1047 ts_tablet_manager.cc:1434] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.035s user 0.004s sys 0.003s
I20260810 07:53:39.764498 4254 raft_consensus.cc:493] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.765105 4254 raft_consensus.cc:515] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.767324 4254 leader_election.cc:290] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:39.767750 4254 raft_consensus.cc:493] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.769766 1071 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P 60781ef3c8b84a8788f43d77e1bb3b19: Bootstrap starting.
I20260810 07:53:39.771623 4206 raft_consensus.cc:493] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:39.772128 4206 raft_consensus.cc:515] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:39.770936 417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:39.774039 4206 leader_election.cc:290] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:39.770071 4254 raft_consensus.cc:515] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.777415 4254 leader_election.cc:290] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:39.779299 417 raft_consensus.cc:2468] T 433404aa29574d819371fe090e670095 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:39.786479 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c075fdf1055477db9f2d56324f12a8a" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.789717 309 leader_election.cc:304] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:39.789994 414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:39.790716 414 raft_consensus.cc:2468] T 5a892c4bc5694118b514cc1baa56601d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:39.790997 4315 raft_consensus.cc:2804] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:39.791395 4315 raft_consensus.cc:493] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:39.791746 4315 raft_consensus.cc:3060] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:39.793573 1047 tablet_bootstrap.cc:492] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:39.801203 373 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c1ddea2d9f3462092fcbaee7774cd01 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae857bd2c124682bbd682d81dc591a5) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:39.800238 403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" is_pre_election: true
I20260810 07:53:39.801899 380 tablet_service.cc:1558] Processing DeleteTablet for tablet afbac73bbeb44af6b39ab23c5cf79eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:57540
I20260810 07:53:39.802178 403 raft_consensus.cc:2468] T a9f5a4321a504ed59e0e2ba21ac801f7 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:39.821601 1047 tablet_bootstrap.cc:654] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.826447 4254 consensus_queue.cc:1048] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:39.828200 573 leader_election.cc:304] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.829294 4219 raft_consensus.cc:2804] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:39.829697 4219 raft_consensus.cc:493] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:39.830086 4219 raft_consensus.cc:3060] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:39.835611 666 raft_consensus.cc:1240] T 04aff252a46447f49c59345035374580 P b5bfa873accc428ab48ac19e1207f88d [term 2 FOLLOWER]: Rejecting Update request from peer 9e545e0597e24d61b7265abed2ccdcfb for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260810 07:53:39.842365 574 leader_election.cc:304] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:39.844043 4042 raft_consensus.cc:2749] T e38ab80cd5c441f3ba20bf00577b223c P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:39.849274 1064 tablet_bootstrap.cc:492] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.849704 1064 ts_tablet_manager.cc:1403] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.250s user 0.018s sys 0.019s
I20260810 07:53:39.851974 1064 raft_consensus.cc:359] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.852630 1064 raft_consensus.cc:385] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.852924 1064 raft_consensus.cc:740] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.858620 573 leader_election.cc:304] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:39.859710 1064 consensus_queue.cc:260] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.860496 4042 raft_consensus.cc:2804] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:39.860936 4042 raft_consensus.cc:493] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:39.861346 4042 raft_consensus.cc:3060] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:39.869240 4042 raft_consensus.cc:515] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.874569 4217 raft_consensus.cc:493] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:39.875080 4217 raft_consensus.cc:515] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.876801 1064 ts_tablet_manager.cc:1434] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260810 07:53:39.877094 875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04aff252a46447f49c59345035374580" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:39.877235 4217 leader_election.cc:290] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:39.877852 1064 tablet_bootstrap.cc:492] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:39.891871 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.893489 1064 tablet_bootstrap.cc:654] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.893630 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02aece0af740828f4009706445e60c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:39.894134 889 raft_consensus.cc:2468] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
I20260810 07:53:39.907783 1064 tablet_bootstrap.cc:492] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:39.908254 1064 ts_tablet_manager.cc:1403] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.003s
I20260810 07:53:39.863548 4361 tablet.cc:1724] T b7bbccbecf884768b3b171d9bd901b9b P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:39.875082 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "666f36dafa92421cae47c94593ed8899" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:39.914443 1064 raft_consensus.cc:359] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.915125 1064 raft_consensus.cc:385] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:39.915385 1064 raft_consensus.cc:740] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:39.915966 1064 consensus_queue.cc:260] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.917465 1064 ts_tablet_manager.cc:1434] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260810 07:53:39.918989 574 leader_election.cc:304] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:39.941475 4368 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:39.942096 4368 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:39.944327 4368 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
W20260810 07:53:39.946094 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 290051us
I20260810 07:53:39.946324 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 416 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:39.948326 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 416 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:39.949926 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:39.954654 1071 tablet_bootstrap.cc:654] T 666f36dafa92421cae47c94593ed8899 P 60781ef3c8b84a8788f43d77e1bb3b19: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:39.964942 4368 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260810 07:53:39.982601 4315 raft_consensus.cc:515] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.984787 4315 leader_election.cc:290] T 44ff256b314e479d981333d9cbfc1656 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.000326 4302 raft_consensus.cc:493] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.001132 4302 raft_consensus.cc:515] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.003906 4302 leader_election.cc:290] T 94ced893d6804aa395b73f1f5ed085f9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.010787 4238 raft_consensus.cc:493] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.012192 4238 raft_consensus.cc:515] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.011317 4246 raft_consensus.cc:493] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.014612 4238 leader_election.cc:290] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.014858 4246 raft_consensus.cc:515] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.016822 4246 leader_election.cc:290] T b305a232550e4d4babb19093ea1163f0 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:40.016940 308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.66:37655, user_credentials={real_user=slave}} blocked reactor thread for 70172.9us
I20260810 07:53:40.011499 4319 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.018872 4319 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.022153 4319 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.029333 4358 raft_consensus.cc:2749] T 1f8941c49c0a4fba9f7de2aabb790901 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:39.894627 4219 raft_consensus.cc:515] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:39.910082 1047 tablet_bootstrap.cc:492] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:40.030066 792 leader_election.cc:304] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:40.030488 1047 ts_tablet_manager.cc:1403] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.237s user 0.006s sys 0.008s
I20260810 07:53:40.032491 308 leader_election.cc:304] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:40.036635 1047 raft_consensus.cc:359] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.037149 4217 raft_consensus.cc:2804] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:40.037554 1047 raft_consensus.cc:385] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.037578 4217 raft_consensus.cc:493] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:40.037876 1047 raft_consensus.cc:740] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.038013 4217 raft_consensus.cc:3060] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.038691 1047 consensus_queue.cc:260] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.039795 4212 raft_consensus.cc:493] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.040244 4212 raft_consensus.cc:515] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.042284 4212 leader_election.cc:290] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.045068 1047 ts_tablet_manager.cc:1434] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260810 07:53:40.045823 4206 consensus_queue.cc:1048] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:40.046363 4042 leader_election.cc:290] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.051062 1047 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:40.052595 4096 raft_consensus.cc:2804] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:53:40.053267 4096 raft_consensus.cc:697] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [term 2 LEADER]: Becoming Leader. State: Replica: 60781ef3c8b84a8788f43d77e1bb3b19, State: Running, Role: LEADER
I20260810 07:53:40.054486 4096 consensus_queue.cc:237] T 04aff252a46447f49c59345035374580 P 60781ef3c8b84a8788f43d77e1bb3b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:40.088104 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 169187us
I20260810 07:53:40.089324 4217 raft_consensus.cc:515] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.090621 32463 catalog_manager.cc:5719] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:40.091321 856 tablet_service.cc:1511] Processing CreateTablet for tablet 4ecc6870602043269daa61cb173108b3 (DEFAULT_TABLE table=default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.092326 32463 catalog_manager.cc:5719] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:40.093747 32463 catalog_manager.cc:5719] T e0a74b22a6864180956e763d2b6b6de6 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:40.094724 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet e0a74b22a6864180956e763d2b6b6de6 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]). Expected version 1 got 0
I20260810 07:53:40.095417 32463 catalog_manager.cc:5719] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
W20260810 07:53:40.097177 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 49479.4us
I20260810 07:53:40.093502 666 raft_consensus.cc:3060] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.094997 630 tablet_service.cc:1511] Processing CreateTablet for tablet 4ecc6870602043269daa61cb173108b3 (DEFAULT_TABLE table=default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.095999 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f8941c49c0a4fba9f7de2aabb790901" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.097952 792 leader_election.cc:304] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:40.098163 1047 tablet_bootstrap.cc:654] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.099799 667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02aece0af740828f4009706445e60c" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.100338 574 leader_election.cc:304] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:40.100447 667 raft_consensus.cc:2468] T 8c02aece0af740828f4009706445e60c P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60781ef3c8b84a8788f43d77e1bb3b19 in term 0.
W20260810 07:53:40.101471 574 consensus_peers.cc:597] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d -> Peer 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): Couldn't send request to peer 9e545e0597e24d61b7265abed2ccdcfb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d5ab03facf8477eada5d1ac3ec7f5ef. This is attempt 11: this message will repeat every 5th retry.
I20260810 07:53:40.091578 1071 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P 60781ef3c8b84a8788f43d77e1bb3b19: No bootstrap required, opened a new log
I20260810 07:53:40.096426 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]). Expected version 1 got 0
I20260810 07:53:40.105440 1071 ts_tablet_manager.cc:1403] T 666f36dafa92421cae47c94593ed8899 P 60781ef3c8b84a8788f43d77e1bb3b19: Time spent bootstrapping tablet: real 0.336s user 0.011s sys 0.002s
I20260810 07:53:40.105820 32463 catalog_manager.cc:5719] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:40.107390 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]). Expected version 2 got 0
I20260810 07:53:40.108039 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]). Expected version 1 got 0
I20260810 07:53:40.108960 32463 catalog_manager.cc:5719] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:40.110749 32462 catalog_manager.cc:6006] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a: Sending DeleteTablet for 3 replicas of tablet 2905f0597c07439989a00a213fc2194f
I20260810 07:53:40.112313 32462 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:43806 (request call id 389) took 1637 ms. Trace:
I20260810 07:53:40.114854 792 leader_election.cc:304] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:40.116022 1064 tablet_bootstrap.cc:492] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:40.112627 32462 rpcz_store.cc:276] 0810 07:53:38.475009 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:38.475252 (+ 243us) service_pool.cc:224] Handling call
0810 07:53:38.477315 (+ 2063us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0810 07:53:38.481482 (+ 4167us) catalog_manager.cc:2841] Modifying in-memory table state
0810 07:53:38.481604 (+ 122us) catalog_manager.cc:2850] Locking tablets
0810 07:53:38.482158 (+ 554us) catalog_manager.cc:2864] Removing table and tablets from system table
0810 07:53:39.373472 (+891314us) catalog_manager.cc:2882] Removing table from by-name map
0810 07:53:39.373737 (+ 265us) catalog_manager.cc:2898] Committing in-memory state
0810 07:53:40.110663 (+736926us) catalog_manager.cc:2902] Aborting table tasks
0810 07:53:40.112215 (+ 1552us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:39.311374 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:39.312106 (+ 732us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:39.312161 (+ 55us) write_op.cc:435] Acquired schema lock
0810 07:53:39.312195 (+ 34us) tablet.cc:662] Decoding operations
0810 07:53:39.331264 (+ 19069us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:39.331317 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:39.331358 (+ 41us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:53:39.331706 (+ 348us) tablet.cc:701] Row locks acquired
0810 07:53:39.331730 (+ 24us) write_op.cc:260] PREPARE: finished
0810 07:53:39.331866 (+ 136us) write_op.cc:270] Start()
0810 07:53:39.331984 (+ 118us) write_op.cc:276] Timestamp: P: 1786348419331801 usec, L: 0
0810 07:53:39.332003 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:39.332635 (+ 632us) log.cc:844] Serialized 731 byte log entry
0810 07:53:39.371393 (+ 38758us) op_driver.cc:464] REPLICATION: finished
0810 07:53:39.371667 (+ 274us) write_op.cc:301] APPLY: starting
0810 07:53:39.371732 (+ 65us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:39.371855 (+ 123us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:39.371971 (+ 116us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:39.372354 (+ 383us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:39.372591 (+ 237us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0810 07:53:39.372672 (+ 81us) write_op.cc:312] APPLY: finished
0810 07:53:39.372959 (+ 287us) log.cc:844] Serialized 29 byte log entry
0810 07:53:39.373139 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:39.373212 (+ 73us) write_op.cc:454] Released schema lock
0810 07:53:39.373246 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0810 07:53:39.373974 (+ 728us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:39.374011 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":106,"apply.run_cpu_time_us":1774,"apply.run_wall_time_us":2254,"num_ops":2,"prepare.queue_time_us":827362,"prepare.run_cpu_time_us":3371,"prepare.run_wall_time_us":37784,"raft.queue_time_us":6923,"raft.run_cpu_time_us":993,"raft.run_wall_time_us":2633,"replication_time_us":39251}]]}
I20260810 07:53:40.091416 4217 leader_election.cc:290] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.116065 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ecc6870602043269daa61cb173108b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.119520 32301 master-stress-test.cc:581] Killed test thread 7/10
I20260810 07:53:40.119777 32301 master-stress-test.cc:579] Killing test thread 8/10
I20260810 07:53:40.120114 32301 master-stress-test.cc:581] Killed test thread 8/10
I20260810 07:53:40.120337 32301 master-stress-test.cc:579] Killing test thread 9/10
I20260810 07:53:40.120658 32301 master-stress-test.cc:581] Killed test thread 9/10
I20260810 07:53:40.122076 32301 master-stress-test.cc:579] Killing test thread 10/10
I20260810 07:53:40.122377 32301 master-stress-test.cc:581] Killed test thread 10/10
I20260810 07:53:40.122731 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 32714
I20260810 07:53:39.760550 4305 ts_tablet_manager.cc:1929] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:40.133912 4305 log.cc:1199] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/5c1ddea2d9f3462092fcbaee7774cd01
I20260810 07:53:40.134124 630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ecc6870602043269daa61cb173108b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.137949 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94ced893d6804aa395b73f1f5ed085f9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.140767 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ff256b314e479d981333d9cbfc1656" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:40.144917 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.146070 677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.147500 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.156997 4305 ts_tablet_manager.cc:1950] T 5c1ddea2d9f3462092fcbaee7774cd01 P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:40.184088 622 tablet_service.cc:1511] Processing CreateTablet for tablet 613d1c158c7c4f2f8626536292d00fd1 (DEFAULT_TABLE table=default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.119518 4206 raft_consensus.cc:2749] T 2a8f390b0db64e1d96c3d8686274cbbf P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:40.148641 4217 raft_consensus.cc:2804] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:40.193307 864 tablet_service.cc:1511] Processing CreateTablet for tablet 505e2f7a25a447c3bf19b310aca1702c (DEFAULT_TABLE table=default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.194478 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505e2f7a25a447c3bf19b310aca1702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.201989 855 tablet_service.cc:1511] Processing CreateTablet for tablet 613d1c158c7c4f2f8626536292d00fd1 (DEFAULT_TABLE table=default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.203191 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 613d1c158c7c4f2f8626536292d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.204481 1064 tablet_bootstrap.cc:654] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.208137 852 tablet_service.cc:1558] Processing DeleteTablet for tablet 2905f0597c07439989a00a213fc2194f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:40.195132 4217 raft_consensus.cc:493] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:40.213145 4217 raft_consensus.cc:3060] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.213390 862 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0f23d5212a46cc8f731d9c08019858 (DEFAULT_TABLE table=default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.214658 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0f23d5212a46cc8f731d9c08019858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.185190 639 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0f23d5212a46cc8f731d9c08019858 (DEFAULT_TABLE table=default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.185222 645 tablet_service.cc:1558] Processing DeleteTablet for tablet 2905f0597c07439989a00a213fc2194f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:40.229912 849 tablet_service.cc:1511] Processing CreateTablet for tablet 38f8faef6d284c98bf5fd9f7c6f9c443 (DEFAULT_TABLE table=default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.231366 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f8faef6d284c98bf5fd9f7c6f9c443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.239271 646 tablet_service.cc:1511] Processing CreateTablet for tablet 38f8faef6d284c98bf5fd9f7c6f9c443 (DEFAULT_TABLE table=default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.240232 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 613d1c158c7c4f2f8626536292d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.171192 4042 raft_consensus.cc:2749] T f6cd26a04d3c417d87091f05bc5cb5cd P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:40.258685 624 tablet_service.cc:1511] Processing CreateTablet for tablet 505e2f7a25a447c3bf19b310aca1702c (DEFAULT_TABLE table=default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.259943 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505e2f7a25a447c3bf19b310aca1702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.259831 4384 raft_consensus.cc:493] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.266052 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f8faef6d284c98bf5fd9f7c6f9c443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.269133 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0f23d5212a46cc8f731d9c08019858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.270522 4362 raft_consensus.cc:493] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.271281 4362 raft_consensus.cc:515] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:40.282040 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 167251us
I20260810 07:53:40.283571 4362 leader_election.cc:290] T 91c2e1bcd13a4c22b2141949b2d73188 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.284598 4305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 384) took 1007 ms. Trace:
I20260810 07:53:40.285149 4305 rpcz_store.cc:276] 0810 07:53:39.277478 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.277615 (+ 137us) service_pool.cc:224] Handling call
0810 07:53:39.607931 (+330316us) ts_tablet_manager.cc:1151] Deleting tablet 5c1ddea2d9f3462092fcbaee7774cd01
0810 07:53:39.607997 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:39.706137 (+ 98140us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:39.760415 (+ 54278us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.284482 (+524067us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117921,"tablet-delete.queue_time_us":329576}
I20260810 07:53:40.293417 4361 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:57706 (request call id 382) took 1229 ms. Trace:
I20260810 07:53:40.306200 1047 tablet_bootstrap.cc:492] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:40.261013 4384 raft_consensus.cc:515] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.317720 4384 leader_election.cc:290] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.313720 4217 raft_consensus.cc:515] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.323907 4217 leader_election.cc:290] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.306806 1047 ts_tablet_manager.cc:1403] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.256s user 0.015s sys 0.002s
I20260810 07:53:40.301236 4219 leader_election.cc:290] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.344406 4305 tablet_replica.cc:333] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:40.345836 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 5c1ddea2d9f3462092fcbaee7774cd01 (table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]) successfully deleted
I20260810 07:53:40.294036 4361 rpcz_store.cc:276] 0810 07:53:39.063686 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.064664 (+ 978us) service_pool.cc:224] Handling call
0810 07:53:40.293300 (+1228636us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:39.066534 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0810 07:53:39.067212 (+ 678us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0810 07:53:39.067261 (+ 49us) alter_schema_op.cc:66] Acquired schema lock
0810 07:53:39.067418 (+ 157us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0810 07:53:39.067613 (+ 195us) alter_schema_op.cc:128] START. Timestamp: 1786348419067476
0810 07:53:39.067644 (+ 31us) op_driver.cc:348] REPLICATION: starting
0810 07:53:39.068267 (+ 623us) log.cc:844] Serialized 173 byte log entry
0810 07:53:39.848662 (+780395us) op_driver.cc:464] REPLICATION: finished
0810 07:53:39.863425 (+ 14763us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0810 07:53:39.912007 (+ 48582us) log.cc:844] Serialized 5 byte log entry
0810 07:53:40.155480 (+243473us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.240741 (+ 85261us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4800002dd8. stack: 00007f22b562bcf3 00007f22b562bbf0 00007f22b51d9574 0000000000516de4 0000000000516d79 00007f22bd787211 00007f22bd77e2a7 00007f22bd77e13a 00007f22bd87507f 00007f22bd874d47 00007f22bd88936c 00007f22bd889318 00007f22bd889298 00007f22bd889253 00007f22bd8884fc 00007f22bcc27046
0810 07:53:40.260276 (+ 19535us) log.cc:844] Serialized 14 byte log entry
0810 07:53:40.292877 (+ 32601us) alter_schema_op.cc:73] Released schema lock
0810 07:53:40.293081 (+ 204us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14497,"mutex_wait_us":14793,"prepare.queue_time_us":249,"prepare.run_cpu_time_us":4673,"prepare.run_wall_time_us":94019,"raft.queue_time_us":17025,"raft.run_cpu_time_us":1125,"raft.run_wall_time_us":1124,"replication_time_us":780871,"spinlock_wait_cycles":69739776,"thread_start_us":117912,"threads_started":2,"wal-append.queue_time_us":76758}]]}
W20260810 07:53:40.364530 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 144645us
I20260810 07:53:40.365485 574 leader_election.cc:304] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5bfa873accc428ab48ac19e1207f88d; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb
I20260810 07:53:40.380278 573 leader_election.cc:304] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:40.381137 4384 consensus_queue.cc:1048] T 8ee78955a68a451f81e01a62a57aa4a4 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:40.381984 4212 raft_consensus.cc:2804] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:40.382323 4212 raft_consensus.cc:493] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:40.382596 4212 raft_consensus.cc:3060] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.385916 4304 ts_tablet_manager.cc:1929] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:40.386471 4304 log.cc:1199] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/5c1ddea2d9f3462092fcbaee7774cd01
I20260810 07:53:40.387459 4304 ts_tablet_manager.cc:1950] T 5c1ddea2d9f3462092fcbaee7774cd01 P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:40.391103 4212 raft_consensus.cc:515] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.393810 4212 leader_election.cc:290] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.398128 1064 tablet_bootstrap.cc:492] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:40.398614 1064 ts_tablet_manager.cc:1403] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.285s user 0.013s sys 0.007s
W20260810 07:53:40.403254 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 120198us
I20260810 07:53:40.404953 792 leader_election.cc:304] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60781ef3c8b84a8788f43d77e1bb3b19, 9e545e0597e24d61b7265abed2ccdcfb; no voters: b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:40.405781 4389 raft_consensus.cc:2804] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:40.406162 4389 raft_consensus.cc:493] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:40.406543 4389 raft_consensus.cc:3060] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:40.413769 32463 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 04aff252a46447f49c59345035374580 consensus state has no leader
I20260810 07:53:40.414288 4389 raft_consensus.cc:515] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:40.414927 32463 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet a6077111ffb2411892b62d806ea3c85b consensus state has no leader
I20260810 07:53:40.427793 1047 raft_consensus.cc:359] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.430063 4396 raft_consensus.cc:493] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.428740 1047 raft_consensus.cc:385] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.431159 1047 raft_consensus.cc:740] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.432060 1047 consensus_queue.cc:260] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:40.437431 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 104) took 1362 ms (client timeout 1000 ms). Trace:
I20260810 07:53:40.438176 639 tablet_service.cc:1558] Processing DeleteTablet for tablet 705fc015928140a58db979445068365f with delete_type TABLET_DATA_DELETED (replaced by tablet fce424ac9b4048c4a988f0fa48acbf46) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:40.430616 4396 raft_consensus.cc:515] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.439186 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d5ab03facf8477eada5d1ac3ec7f5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:05 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:40.441020 4396 leader_election.cc:290] T b305a232550e4d4babb19093ea1163f0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:40.443363 1047 ts_tablet_manager.cc:1434] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.117s user 0.006s sys 0.000s
I20260810 07:53:40.444461 1047 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:40.455701 4358 raft_consensus.cc:2749] T a6077111ffb2411892b62d806ea3c85b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:40.461341 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 04aff252a46447f49c59345035374580 consensus state has no leader
I20260810 07:53:40.432545 4304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 330) took 1155 ms. Trace:
I20260810 07:53:40.464906 4304 rpcz_store.cc:276] 0810 07:53:39.277190 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.277335 (+ 145us) service_pool.cc:224] Handling call
0810 07:53:39.701738 (+424403us) ts_tablet_manager.cc:1151] Deleting tablet 5c1ddea2d9f3462092fcbaee7774cd01
0810 07:53:39.702063 (+ 325us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:40.251220 (+549157us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.385766 (+134546us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.432470 (+ 46704us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":422981}
I20260810 07:53:40.466373 4389 leader_election.cc:290] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.467041 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 5c1ddea2d9f3462092fcbaee7774cd01 (table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]) successfully deleted
I20260810 07:53:40.468283 4304 tablet_replica.cc:333] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
W20260810 07:53:40.457072 792 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.139.65:39809, user_credentials={real_user=slave}} blocked reactor thread for 52188us
I20260810 07:53:40.438427 1064 raft_consensus.cc:359] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.472182 1064 raft_consensus.cc:385] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.472525 1064 raft_consensus.cc:740] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.473207 1064 consensus_queue.cc:260] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.474654 1064 ts_tablet_manager.cc:1434] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.076s user 0.007s sys 0.000s
I20260810 07:53:40.486284 4399 raft_consensus.cc:493] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.486953 4399 raft_consensus.cc:515] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.489488 1064 tablet_bootstrap.cc:492] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:40.437986 32463 rpcz_store.cc:269] 0810 07:53:39.075277 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.091989 (+ 16712us) service_pool.cc:224] Handling call
0810 07:53:40.252990 (+1161001us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:40.253019 (+ 29us) write_op.cc:454] Released schema lock
0810 07:53:40.437337 (+184318us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:40.174606 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:40.175105 (+ 499us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:40.175133 (+ 28us) write_op.cc:435] Acquired schema lock
0810 07:53:40.175152 (+ 19us) tablet.cc:662] Decoding operations
0810 07:53:40.175888 (+ 736us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:40.175940 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:40.175966 (+ 26us) tablet.cc:685] Acquiring locks for 6 operations
0810 07:53:40.176432 (+ 466us) tablet.cc:701] Row locks acquired
0810 07:53:40.176457 (+ 25us) write_op.cc:260] PREPARE: finished
0810 07:53:40.176592 (+ 135us) write_op.cc:270] Start()
0810 07:53:40.176688 (+ 96us) write_op.cc:276] Timestamp: P: 1786348420176364 usec, L: 0
0810 07:53:40.176707 (+ 19us) op_driver.cc:348] REPLICATION: starting
0810 07:53:40.177390 (+ 683us) log.cc:844] Serialized 2434 byte log entry
0810 07:53:40.188837 (+ 11447us) op_driver.cc:464] REPLICATION: finished
0810 07:53:40.189413 (+ 576us) write_op.cc:301] APPLY: starting
0810 07:53:40.189485 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:40.189663 (+ 178us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:40.189771 (+ 108us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:40.190450 (+ 679us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:40.190698 (+ 248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0810 07:53:40.190767 (+ 69us) write_op.cc:312] APPLY: finished
0810 07:53:40.191094 (+ 327us) log.cc:844] Serialized 53 byte log entry
0810 07:53:40.191239 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:40.191339 (+ 100us) write_op.cc:454] Released schema lock
0810 07:53:40.191374 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":391,"apply.run_cpu_time_us":2171,"apply.run_wall_time_us":2171,"num_ops":6,"prepare.queue_time_us":123,"prepare.run_cpu_time_us":3248,"prepare.run_wall_time_us":30330,"raft.queue_time_us":835,"raft.run_cpu_time_us":1700,"raft.run_wall_time_us":1700,"replication_time_us":11938}]]}
I20260810 07:53:40.493372 4399 leader_election.cc:290] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.518584 666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461d70a71f2c44228f4220e7f5fb9c64" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.519240 666 raft_consensus.cc:2468] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:40.522326 1064 tablet_bootstrap.cc:654] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.528913 4396 raft_consensus.cc:493] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.529393 4396 raft_consensus.cc:515] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.529815 669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e629e89eb734ade82b3dcbf11c971f8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.531284 4396 leader_election.cc:290] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.537892 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb34e5b988b497c8f72a6c7266c0f2d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.558260 1047 tablet_bootstrap.cc:654] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.566138 4254 consensus_queue.cc:1048] T bdc0039db22d4812a1412e10175d2237 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:53:40.601174 1064 tablet_bootstrap.cc:492] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:40.602663 1064 ts_tablet_manager.cc:1403] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.117s user 0.010s sys 0.002s
I20260810 07:53:40.606556 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:40.607196 670 raft_consensus.cc:3060] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.608789 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:40.609485 664 raft_consensus.cc:2393] T 5a892c4bc5694118b514cc1baa56601d P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:40.611217 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e15de89beda46f29a650f93f01d4fda" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.613478 678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:40.614199 678 raft_consensus.cc:2393] T a9f5a4321a504ed59e0e2ba21ac801f7 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:40.618633 4305 raft_consensus.cc:2243] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:40.619436 4305 raft_consensus.cc:2272] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:40.622219 4305 ts_tablet_manager.cc:1916] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:40.629246 1047 tablet_bootstrap.cc:492] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:40.629424 670 raft_consensus.cc:2468] T 05ef1be8eaa84e6bb5586e68268799a0 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:40.632316 4389 raft_consensus.cc:493] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.633028 4389 raft_consensus.cc:515] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.635360 4389 leader_election.cc:290] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.629732 1047 ts_tablet_manager.cc:1403] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.185s user 0.011s sys 0.000s
I20260810 07:53:40.636644 4206 raft_consensus.cc:493] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:40.637231 4206 raft_consensus.cc:515] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:40.638542 1047 raft_consensus.cc:359] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.639143 1047 raft_consensus.cc:385] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.639394 1047 raft_consensus.cc:740] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.639596 4206 leader_election.cc:290] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:40.640201 4206 raft_consensus.cc:493] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.640695 4206 raft_consensus.cc:515] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.641031 1047 consensus_queue.cc:260] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.642797 4206 leader_election.cc:290] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.645077 1064 raft_consensus.cc:359] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.645792 1064 raft_consensus.cc:385] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.655287 1047 ts_tablet_manager.cc:1434] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260810 07:53:40.659399 4305 ts_tablet_manager.cc:1929] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:40.672950 4305 log.cc:1199] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/afbac73bbeb44af6b39ab23c5cf79eed
I20260810 07:53:40.674125 4305 ts_tablet_manager.cc:1950] T afbac73bbeb44af6b39ab23c5cf79eed P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:40.678682 4305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57706 (request call id 385) took 1244 ms. Trace:
I20260810 07:53:40.679145 4305 rpcz_store.cc:276] 0810 07:53:39.434746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.434870 (+ 124us) service_pool.cc:224] Handling call
0810 07:53:40.344203 (+909333us) ts_tablet_manager.cc:1151] Deleting tablet afbac73bbeb44af6b39ab23c5cf79eed
0810 07:53:40.344255 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:40.639212 (+294957us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.659242 (+ 20030us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.677762 (+ 18520us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":273092,"tablet-delete.queue_time_us":908778}
I20260810 07:53:40.680881 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]) successfully deleted
I20260810 07:53:40.681277 4305 tablet_replica.cc:333] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:40.680452 4254 raft_consensus.cc:493] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.681876 4254 raft_consensus.cc:515] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.682739 4305 raft_consensus.cc:2243] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:40.683615 4305 raft_consensus.cc:2272] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:40.685376 4254 leader_election.cc:290] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:40.656553 1047 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:40.686302 4305 ts_tablet_manager.cc:1916] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:40.665875 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.689494 889 raft_consensus.cc:2393] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:40.665197 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.690721 876 raft_consensus.cc:2393] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:40.676085 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.676893 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a892c4bc5694118b514cc1baa56601d" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:40.692792 896 raft_consensus.cc:2393] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:40.646098 1064 raft_consensus.cc:740] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.680372 883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "433404aa29574d819371fe090e670095" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:40.694474 883 raft_consensus.cc:3060] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.694504 1064 consensus_queue.cc:260] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.696425 1064 ts_tablet_manager.cc:1434] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.075s user 0.006s sys 0.000s
I20260810 07:53:40.680014 885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.698233 4304 raft_consensus.cc:2243] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:40.698856 4304 raft_consensus.cc:2272] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:40.701547 4304 ts_tablet_manager.cc:1916] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:40.703575 883 raft_consensus.cc:2468] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:40.731614 4305 ts_tablet_manager.cc:1929] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:40.733918 4305 log.cc:1199] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/2905f0597c07439989a00a213fc2194f
I20260810 07:53:40.713550 1064 tablet_bootstrap.cc:492] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:40.725777 896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461d70a71f2c44228f4220e7f5fb9c64" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.766458 896 raft_consensus.cc:2468] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:40.727284 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:40.767570 892 raft_consensus.cc:2393] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:40.730018 876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b305a232550e4d4babb19093ea1163f0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.741142 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.769078 889 raft_consensus.cc:2393] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:40.756932 4304 ts_tablet_manager.cc:1929] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:53:40.757154 897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd79131856f54395a3c5fb551961d4f7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:40.772472 897 raft_consensus.cc:2468] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:40.773245 4304 log.cc:1199] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/afbac73bbeb44af6b39ab23c5cf79eed
I20260810 07:53:40.774780 4304 ts_tablet_manager.cc:1950] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:40.777092 4304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55008 (request call id 331) took 1350 ms. Trace:
I20260810 07:53:40.777500 4304 rpcz_store.cc:276] 0810 07:53:39.426889 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:39.427040 (+ 151us) service_pool.cc:224] Handling call
0810 07:53:40.467990 (+1040950us) ts_tablet_manager.cc:1151] Deleting tablet afbac73bbeb44af6b39ab23c5cf79eed
0810 07:53:40.468144 (+ 154us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0810 07:53:40.718419 (+250275us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.756393 (+ 37974us) tablet_metadata.cc:665] Metadata flushed
0810 07:53:40.776988 (+ 20595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227003,"tablet-delete.queue_time_us":1040372}
I20260810 07:53:40.779121 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet afbac73bbeb44af6b39ab23c5cf79eed (table default.Table_1af94d6703df4a2eb7e30cdf6301bdce [id=2fc441ad4da747fc828fd952f1107a0e]) successfully deleted
I20260810 07:53:40.775763 1064 tablet_bootstrap.cc:654] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.783828 4304 tablet_replica.cc:333] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: stopping tablet replica
I20260810 07:53:40.784993 4304 raft_consensus.cc:2243] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:53:40.786350 4304 raft_consensus.cc:2272] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:40.791044 4304 ts_tablet_manager.cc:1916] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:40.793073 1047 tablet_bootstrap.cc:654] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.793274 4305 ts_tablet_manager.cc:1950] T 2905f0597c07439989a00a213fc2194f P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
I20260810 07:53:40.759219 792 leader_election.cc:304] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:40.794255 4206 raft_consensus.cc:2764] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 07:53:40.788342 1064 tablet_bootstrap.cc:492] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:40.808362 1064 ts_tablet_manager.cc:1403] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.005s
I20260810 07:53:40.819800 4409 raft_consensus.cc:493] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:40.817291 1064 raft_consensus.cc:359] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.820325 4409 raft_consensus.cc:515] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.821162 1064 raft_consensus.cc:385] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.821928 1064 raft_consensus.cc:740] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.822583 4409 leader_election.cc:290] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:40.821561 792 leader_election.cc:343] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:40.826042 4206 consensus_queue.cc:1059] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:53:40.826941 4389 raft_consensus.cc:3055] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Stepping down as leader of term 1
I20260810 07:53:40.827270 4389 raft_consensus.cc:740] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:40.828081 4389 consensus_queue.cc:260] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.831570 4389 raft_consensus.cc:3060] T 04aff252a46447f49c59345035374580 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:53:40.834556 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 2905f0597c07439989a00a213fc2194f (table default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]) successfully deleted
I20260810 07:53:40.835894 4254 raft_consensus.cc:493] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e545e0597e24d61b7265abed2ccdcfb)
I20260810 07:53:40.836370 4254 raft_consensus.cc:515] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.838742 4254 leader_election.cc:290] T c1826ebc26f04805b70ac5314e48fda9 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
W20260810 07:53:40.848677 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 705fc015928140a58db979445068365f because the tablet was not found. No further retry: Not found: Tablet not found: 705fc015928140a58db979445068365f
W20260810 07:53:40.850108 792 leader_election.cc:343] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:40.851112 1064 consensus_queue.cc:260] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.858629 1047 tablet_bootstrap.cc:492] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:40.859056 1047 ts_tablet_manager.cc:1403] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.203s user 0.010s sys 0.000s
I20260810 07:53:40.867982 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]). Expected version 1 got 0
I20260810 07:53:40.871369 862 tablet_service.cc:1511] Processing CreateTablet for tablet edb1d215d994424a8ff8c6239ffaaed4 (DEFAULT_TABLE table=default.Table_10ebf20223374508adbac3c536614ad3 [id=752ea2a5c0d54b8a837d6b8e7f247d73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.872500 862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb1d215d994424a8ff8c6239ffaaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.872895 792 leader_election.cc:304] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:40.866165 1064 ts_tablet_manager.cc:1434] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.057s user 0.005s sys 0.000s
I20260810 07:53:40.874351 1064 tablet_bootstrap.cc:492] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:40.880363 4206 raft_consensus.cc:2804] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:40.880777 4206 raft_consensus.cc:493] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:40.881011 624 tablet_service.cc:1511] Processing CreateTablet for tablet edb1d215d994424a8ff8c6239ffaaed4 (DEFAULT_TABLE table=default.Table_10ebf20223374508adbac3c536614ad3 [id=752ea2a5c0d54b8a837d6b8e7f247d73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.881201 4206 raft_consensus.cc:3060] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:40.885233 4305 tablet_replica.cc:333] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: stopping tablet replica
I20260810 07:53:40.885169 849 tablet_service.cc:1511] Processing CreateTablet for tablet fce424ac9b4048c4a988f0fa48acbf46 (DEFAULT_TABLE table=default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.886257 624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb1d215d994424a8ff8c6239ffaaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.886389 849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fce424ac9b4048c4a988f0fa48acbf46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.895212 4206 raft_consensus.cc:515] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:40.897248 1047 raft_consensus.cc:359] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.901739 1047 raft_consensus.cc:385] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:40.902765 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet e0a74b22a6864180956e763d2b6b6de6 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]). Expected version 1 got 0
I20260810 07:53:40.903615 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]). Expected version 2 got 0
I20260810 07:53:40.901826 622 tablet_service.cc:1511] Processing CreateTablet for tablet fce424ac9b4048c4a988f0fa48acbf46 (DEFAULT_TABLE table=default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.905033 1047 raft_consensus.cc:740] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:40.905639 622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fce424ac9b4048c4a988f0fa48acbf46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.905809 1047 consensus_queue.cc:260] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.908346 1047 ts_tablet_manager.cc:1434] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.049s user 0.007s sys 0.000s
I20260810 07:53:40.903507 864 tablet_service.cc:1511] Processing CreateTablet for tablet 2d8d439276e5440c8d7082b38a1dd3ed (DEFAULT_TABLE table=default.Table_52b389643dbe4064ba2010a1d2d6c8aa [id=55e0c4c593154fd8bb50dcfe3c932719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.909381 864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d8d439276e5440c8d7082b38a1dd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.909577 639 tablet_service.cc:1511] Processing CreateTablet for tablet 2d8d439276e5440c8d7082b38a1dd3ed (DEFAULT_TABLE table=default.Table_52b389643dbe4064ba2010a1d2d6c8aa [id=55e0c4c593154fd8bb50dcfe3c932719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.914114 32462 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]). Expected version 1 got 0
I20260810 07:53:40.915387 32462 catalog_manager.cc:5719] T 94ced893d6804aa395b73f1f5ed085f9 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
W20260810 07:53:40.904510 792 leader_election.cc:343] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
W20260810 07:53:40.925985 792 leader_election.cc:343] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:40.926918 792 leader_election.cc:304] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:40.932847 1064 tablet_bootstrap.cc:654] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:40.933919 4217 raft_consensus.cc:2804] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:40.934455 4217 raft_consensus.cc:697] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:40.935299 4217 consensus_queue.cc:237] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:40.942330 855 tablet_service.cc:1511] Processing CreateTablet for tablet c592efc3acbf49659ecd7d7acb2881e5 (DEFAULT_TABLE table=default.Table_1315259f9b4e4425a2f99c5125e402bc [id=3b6a4e9c10744d4dbef52ed4a87fc36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.945639 646 tablet_service.cc:1511] Processing CreateTablet for tablet c592efc3acbf49659ecd7d7acb2881e5 (DEFAULT_TABLE table=default.Table_1315259f9b4e4425a2f99c5125e402bc [id=3b6a4e9c10744d4dbef52ed4a87fc36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:40.947849 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c592efc3acbf49659ecd7d7acb2881e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.949479 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf6663cbe7fd43a79265d87c2cc11a62" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.950696 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd79131856f54395a3c5fb551961d4f7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:40.951391 670 raft_consensus.cc:2468] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:40.928855 4206 leader_election.cc:290] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:40.953380 856 tablet_service.cc:1511] Processing CreateTablet for tablet 09e234449dc6407585dc79cfdce766cc (DEFAULT_TABLE table=default.Table_7b1b0b8833e640e1bdddb0a030106d84 [id=d807e002aab64400be9e50e1e447255a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260810 07:53:40.953926 576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:40.959234 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d8d439276e5440c8d7082b38a1dd3ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.980871 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c592efc3acbf49659ecd7d7acb2881e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:40.981552 1047 tablet_bootstrap.cc:492] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:40.982640 856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e234449dc6407585dc79cfdce766cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:41.011533 4304 ts_tablet_manager.cc:1929] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:41.012107 4304 log.cc:1199] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/2905f0597c07439989a00a213fc2194f
I20260810 07:53:41.014279 1064 tablet_bootstrap.cc:492] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:41.027480 646 tablet_service.cc:1511] Processing CreateTablet for tablet 09e234449dc6407585dc79cfdce766cc (DEFAULT_TABLE table=default.Table_7b1b0b8833e640e1bdddb0a030106d84 [id=d807e002aab64400be9e50e1e447255a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:41.028667 646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e234449dc6407585dc79cfdce766cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:53:41.033823 576 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:41.042093 1064 ts_tablet_manager.cc:1403] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.168s user 0.006s sys 0.007s
I20260810 07:53:41.042233 4305 raft_consensus.cc:2243] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:53:41.043089 4305 raft_consensus.cc:2272] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:53:41.032195 4304 ts_tablet_manager.cc:1950] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: Deleting consensus metadata
I20260810 07:53:41.051476 1064 raft_consensus.cc:359] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.052208 1064 raft_consensus.cc:385] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.052484 1064 raft_consensus.cc:740] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.056182 4305 ts_tablet_manager.cc:1916] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:53:41.056526 32451 catalog_manager.cc:5050] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): tablet 2905f0597c07439989a00a213fc2194f (table default.Table_20e00f06bba84d53bf31edd76a1a3940 [id=5a24814c72694994ba7a43374d1d0286]) successfully deleted
I20260810 07:53:41.064121 622 tablet_service.cc:1558] Processing DeleteTablet for tablet 2905f0597c07439989a00a213fc2194f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:57706
I20260810 07:53:41.063911 1064 consensus_queue.cc:260] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.066042 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "461d70a71f2c44228f4220e7f5fb9c64" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:41.066614 664 raft_consensus.cc:3060] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.070093 1047 tablet_bootstrap.cc:654] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.073537 664 raft_consensus.cc:2468] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:41.075575 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]). Expected version 1 got 0
I20260810 07:53:41.076565 4217 raft_consensus.cc:493] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.077060 4217 raft_consensus.cc:515] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.077443 32463 catalog_manager.cc:5719] T a6077111ffb2411892b62d806ea3c85b P 9e545e0597e24d61b7265abed2ccdcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:41.078647 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]). Expected version 1 got 0
I20260810 07:53:41.078913 4217 leader_election.cc:290] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.080188 32463 catalog_manager.cc:5811] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917) does not have the latest schema for tablet 461d70a71f2c44228f4220e7f5fb9c64 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 2 got 0
I20260810 07:53:41.081179 1064 ts_tablet_manager.cc:1434] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260810 07:53:41.084805 4217 raft_consensus.cc:493] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.085449 4217 raft_consensus.cc:515] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.087586 4217 leader_election.cc:290] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.094318 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "666f36dafa92421cae47c94593ed8899" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:41.094939 895 raft_consensus.cc:2468] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:41.096562 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1826ebc26f04805b70ac5314e48fda9" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:41.100612 855 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae857bd2c124682bbd682d81dc591a5 (DEFAULT_TABLE table=default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:41.101953 855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae857bd2c124682bbd682d81dc591a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:41.106233 639 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae857bd2c124682bbd682d81dc591a5 (DEFAULT_TABLE table=default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:53:41.107458 639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae857bd2c124682bbd682d81dc591a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:53:41.109974 1064 tablet_bootstrap.cc:492] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:41.112655 574 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.67:42917, user_credentials={real_user=slave}} blocked reactor thread for 58510.4us
I20260810 07:53:41.117132 1047 tablet_bootstrap.cc:492] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:41.117702 1047 ts_tablet_manager.cc:1403] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.136s user 0.008s sys 0.003s
W20260810 07:53:41.131091 32462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 105) took 881 ms (client timeout 1000 ms). Trace:
I20260810 07:53:41.133890 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2cc9ff5b3344dc39e2e560d63f0ce92" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
W20260810 07:53:41.131577 32462 rpcz_store.cc:269] 0810 07:53:40.250115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:40.250392 (+ 277us) service_pool.cc:224] Handling call
0810 07:53:40.392504 (+142112us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b5400030554. stack: 00007f30c8863cf3 00007f30c8863bf0 00007f30c8411574 0000000000516de4 0000000000516d79 00007f30cf965470 00007f30cfa0431a 00007f30d176e05c 00007f30d186ac7e 00007f30cf418fb4 00007f30cf418f42 00007f30cf418e71 00007f30cf418dec 00007f30cf4180a2 00007f30cd6c607c 00007f30cd6c54b6
0810 07:53:41.009104 (+616600us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:41.009131 (+ 27us) write_op.cc:454] Released schema lock
0810 07:53:41.131017 (+121886us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:53:40.969638 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:53:40.970248 (+ 610us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:53:40.970279 (+ 31us) write_op.cc:435] Acquired schema lock
0810 07:53:40.970296 (+ 17us) tablet.cc:662] Decoding operations
0810 07:53:40.970622 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:53:40.970675 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0810 07:53:40.970703 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0810 07:53:40.970906 (+ 203us) tablet.cc:701] Row locks acquired
0810 07:53:40.970921 (+ 15us) write_op.cc:260] PREPARE: finished
0810 07:53:40.971066 (+ 145us) write_op.cc:270] Start()
0810 07:53:40.971169 (+ 103us) write_op.cc:276] Timestamp: P: 1786348420970977 usec, L: 0
0810 07:53:40.971185 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:53:40.971816 (+ 631us) log.cc:844] Serialized 533 byte log entry
0810 07:53:40.987279 (+ 15463us) op_driver.cc:464] REPLICATION: finished
0810 07:53:40.987660 (+ 381us) write_op.cc:301] APPLY: starting
0810 07:53:40.987726 (+ 66us) tablet.cc:1367] starting BulkCheckPresence
0810 07:53:40.987841 (+ 115us) tablet.cc:1370] finished BulkCheckPresence
0810 07:53:40.987938 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:53:40.988125 (+ 187us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:53:40.988276 (+ 151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0810 07:53:40.988326 (+ 50us) write_op.cc:312] APPLY: finished
0810 07:53:40.988560 (+ 234us) log.cc:844] Serialized 23 byte log entry
0810 07:53:40.988762 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:40.988859 (+ 97us) write_op.cc:454] Released schema lock
0810 07:53:40.988893 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":17634,"reported_tablets":16,"spinlock_wait_cycles":111175424,"child_traces":[["op",{"apply.queue_time_us":224,"apply.run_cpu_time_us":1428,"apply.run_wall_time_us":1427,"num_ops":1,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":2539,"prepare.run_wall_time_us":2540,"raft.queue_time_us":365,"raft.run_cpu_time_us":962,"raft.run_wall_time_us":961,"replication_time_us":15977}]]}
I20260810 07:53:41.137231 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5a69fed75047f18522bfee894c769b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.147820 4217 raft_consensus.cc:493] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.148322 4217 raft_consensus.cc:515] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.150188 4217 leader_election.cc:290] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:41.151099 792 leader_election.cc:343] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Not found: Tablet not found: 8d92e0a84839427099a9da06e2b970ce
I20260810 07:53:41.146638 4450 tablet.cc:1724] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:41.158087 1064 tablet_bootstrap.cc:654] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:41.152489 792 leader_election.cc:343] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:41.162137 792 leader_election.cc:304] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.167006 4217 raft_consensus.cc:2804] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.167474 4217 raft_consensus.cc:493] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.167881 4217 raft_consensus.cc:3060] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.120474 1047 raft_consensus.cc:359] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.173944 1047 raft_consensus.cc:385] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.174279 1047 raft_consensus.cc:740] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.174929 1047 consensus_queue.cc:260] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.175755 664 raft_consensus.cc:2468] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 0.
I20260810 07:53:41.178094 792 leader_election.cc:304] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.180034 4206 raft_consensus.cc:2804] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:41.180487 4206 raft_consensus.cc:697] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:41.181294 4206 consensus_queue.cc:237] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.179080 4217 raft_consensus.cc:515] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.185746 856 tablet_service.cc:1558] Processing DeleteTablet for tablet 2905f0597c07439989a00a213fc2194f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:53:38 UTC) from {username='slave'} at 127.0.0.1:55008
I20260810 07:53:41.189127 574 leader_election.cc:304] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.189836 4408 raft_consensus.cc:2804] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:53:41.189867 792 leader_election.cc:343] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:41.189951 574 leader_election.cc:304] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.190322 4408 raft_consensus.cc:697] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:41.190864 4449 raft_consensus.cc:2764] T 461d70a71f2c44228f4220e7f5fb9c64 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 07:53:41.191094 4408 consensus_queue.cc:237] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.191500 574 leader_election.cc:304] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.192164 4449 raft_consensus.cc:2804] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.192476 4449 raft_consensus.cc:493] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.192827 4449 raft_consensus.cc:3060] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.194197 4217 leader_election.cc:290] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.207031 574 leader_election.cc:304] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
W20260810 07:53:41.188153 32451 catalog_manager.cc:5017] TS 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917): delete failed for tablet 2905f0597c07439989a00a213fc2194f because the tablet was not found. No further retry: Not found: Tablet not found: 2905f0597c07439989a00a213fc2194f
I20260810 07:53:41.212517 741 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:53:41.214036 4254 raft_consensus.cc:2804] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.214504 4254 raft_consensus.cc:493] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.214897 4254 raft_consensus.cc:3060] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.216292 4305 ts_tablet_manager.cc:1929] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:53:41.210302 792 leader_election.cc:304] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.208257 4449 raft_consensus.cc:515] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:41.221411 32463 catalog_manager.cc:5911] Failed to send AlterTable RPC for tablet 461d70a71f2c44228f4220e7f5fb9c64 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 461d70a71f2c44228f4220e7f5fb9c64 consensus state has no leader
I20260810 07:53:41.221946 4217 raft_consensus.cc:2804] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.222396 4217 raft_consensus.cc:493] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.222805 4217 raft_consensus.cc:3060] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.238422 961 heartbeater.cc:499] Master 127.31.139.125:46845 was elected leader, sending a full tablet report...
I20260810 07:53:41.217193 1047 ts_tablet_manager.cc:1434] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.099s user 0.006s sys 0.000s
I20260810 07:53:41.232848 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb366297938415382cf6555ed052956" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.252358 1047 tablet_bootstrap.cc:492] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:41.234153 664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd79131856f54395a3c5fb551961d4f7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:41.253252 664 raft_consensus.cc:2393] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e545e0597e24d61b7265abed2ccdcfb in current term 1: Already voted for candidate b5bfa873accc428ab48ac19e1207f88d in this term.
I20260810 07:53:41.227468 4449 leader_election.cc:290] T dd79131856f54395a3c5fb551961d4f7 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.216693 4305 log.cc:1199] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/2d5ab03facf8477eada5d1ac3ec7f5ef
I20260810 07:53:41.241796 4461 tablet.cc:1724] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:41.256536 4305 ts_tablet_manager.cc:1950] T 2d5ab03facf8477eada5d1ac3ec7f5ef P b5bfa873accc428ab48ac19e1207f88d: Deleting consensus metadata
W20260810 07:53:41.275837 792 leader_election.cc:343] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:41.281555 889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd79131856f54395a3c5fb551961d4f7" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:41.282284 889 raft_consensus.cc:2393] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5bfa873accc428ab48ac19e1207f88d in current term 1: Already voted for candidate 9e545e0597e24d61b7265abed2ccdcfb in this term.
I20260810 07:53:41.282940 4254 raft_consensus.cc:515] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.288017 4399 raft_consensus.cc:493] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.288553 4399 raft_consensus.cc:515] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:41.289773 574 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:41.288408 4470 tablet.cc:1724] T 9b4339a26ad540f4b385f69fd0dcdc5d P b5bfa873accc428ab48ac19e1207f88d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260810 07:53:41.290534 4399 leader_election.cc:290] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.295073 4254 leader_election.cc:290] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:41.297637 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a86401e669fa4358ae33ce2249b3a98e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.303930 1047 tablet_bootstrap.cc:654] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.276641 1064 tablet_bootstrap.cc:492] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:41.309140 1064 ts_tablet_manager.cc:1403] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.199s user 0.011s sys 0.008s
I20260810 07:53:41.321962 32450 catalog_manager.cc:5050] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): tablet 2d5ab03facf8477eada5d1ac3ec7f5ef (table default.Table_ae927dcca7ec42b99eee31143de3bf0b [id=b70f05a8651b408393dcb8a5cae4507d]) successfully deleted
W20260810 07:53:41.322718 32450 catalog_manager.cc:5017] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): delete failed for tablet 2905f0597c07439989a00a213fc2194f because the tablet was not found. No further retry: Not found: Tablet not found: 2905f0597c07439989a00a213fc2194f
I20260810 07:53:41.311589 1064 raft_consensus.cc:359] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.346446 1047 tablet_bootstrap.cc:492] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:41.346873 1047 ts_tablet_manager.cc:1403] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.095s user 0.005s sys 0.006s
I20260810 07:53:41.350419 1047 raft_consensus.cc:359] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.351043 1047 raft_consensus.cc:385] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.351292 1047 raft_consensus.cc:740] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.352070 1047 consensus_queue.cc:260] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.354105 1047 ts_tablet_manager.cc:1434] T b2cc9ff5b3344dc39e2e560d63f0ce92 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260810 07:53:41.354997 1047 tablet_bootstrap.cc:492] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:41.384848 1064 raft_consensus.cc:385] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.385198 1064 raft_consensus.cc:740] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.386080 1064 consensus_queue.cc:260] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.341431 4217 raft_consensus.cc:515] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.316977 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "666f36dafa92421cae47c94593ed8899" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:41.388777 1047 tablet_bootstrap.cc:654] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.388698 895 raft_consensus.cc:3060] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.357482 4399 raft_consensus.cc:493] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.392966 4399 raft_consensus.cc:515] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.394979 4399 leader_election.cc:290] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:41.370442 800 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:41.397375 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20d46d62cdc48669d7474fae00660d7" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.399837 4217 leader_election.cc:290] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Requested vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.403280 895 raft_consensus.cc:2468] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:41.404207 1064 ts_tablet_manager.cc:1434] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.095s user 0.005s sys 0.001s
I20260810 07:53:41.405156 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5a69fed75047f18522bfee894c769b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d"
I20260810 07:53:41.405220 4217 raft_consensus.cc:493] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.405673 671 raft_consensus.cc:3060] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.405743 4217 raft_consensus.cc:515] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.407110 4399 raft_consensus.cc:493] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.407648 4399 raft_consensus.cc:515] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.409705 4399 leader_election.cc:290] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:41.409696 792 leader_election.cc:343] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655): Illegal state: must be running to vote when last-logged opid is not known
I20260810 07:53:41.410777 4389 raft_consensus.cc:493] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.411185 1047 tablet_bootstrap.cc:492] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:41.411240 4389 raft_consensus.cc:515] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.411645 1047 ts_tablet_manager.cc:1403] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.057s user 0.010s sys 0.001s
I20260810 07:53:41.412130 4433 raft_consensus.cc:493] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.412636 4433 raft_consensus.cc:515] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:41.413234 4389 leader_election.cc:290] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.414716 4433 leader_election.cc:290] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.414790 1047 raft_consensus.cc:359] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.415417 1047 raft_consensus.cc:385] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.415676 1047 raft_consensus.cc:740] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.415831 574 leader_election.cc:304] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.416646 4466 raft_consensus.cc:2804] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:41.417502 4466 raft_consensus.cc:697] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:41.418578 4466 consensus_queue.cc:237] T 666f36dafa92421cae47c94593ed8899 P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.419591 1047 consensus_queue.cc:260] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.421236 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d1c8d270f744ee9849475c9128414b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.423856 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0a74b22a6864180956e763d2b6b6de6" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.427763 4217 leader_election.cc:290] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.421445 1047 ts_tablet_manager.cc:1434] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260810 07:53:41.437537 1047 tablet_bootstrap.cc:492] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:41.442108 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe1a943230432dbe533e7b3671172b" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.442822 1047 tablet_bootstrap.cc:654] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.445698 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "999f3e79ee4b4c368b60b202735c2556" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.451946 671 raft_consensus.cc:2468] T 0e5a69fed75047f18522bfee894c769b P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e545e0597e24d61b7265abed2ccdcfb in term 1.
I20260810 07:53:41.458570 792 leader_election.cc:304] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.469592 1047 tablet_bootstrap.cc:492] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:41.470098 1047 ts_tablet_manager.cc:1403] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260810 07:53:41.472615 1047 raft_consensus.cc:359] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.473448 1047 raft_consensus.cc:385] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.473712 1047 raft_consensus.cc:740] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.474246 4217 raft_consensus.cc:2804] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:41.474828 4217 raft_consensus.cc:697] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Running, Role: LEADER
I20260810 07:53:41.475740 4217 consensus_queue.cc:237] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.481564 1047 consensus_queue.cc:260] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.489996 1047 ts_tablet_manager.cc:1434] T 4bb34e5b988b497c8f72a6c7266c0f2d P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260810 07:53:41.491436 1064 tablet_bootstrap.cc:492] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:41.510670 1064 tablet_bootstrap.cc:654] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.518862 4466 raft_consensus.cc:493] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.519596 4466 raft_consensus.cc:515] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.523038 4466 leader_election.cc:290] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.539175 1064 tablet_bootstrap.cc:492] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:41.539594 1064 ts_tablet_manager.cc:1403] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.048s user 0.012s sys 0.000s
I20260810 07:53:41.542074 1064 raft_consensus.cc:359] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.542696 1064 raft_consensus.cc:385] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.542963 1064 raft_consensus.cc:740] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.543550 1064 consensus_queue.cc:260] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.545190 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e15de89beda46f29a650f93f01d4fda" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:41.545845 892 raft_consensus.cc:2468] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:41.546919 574 leader_election.cc:304] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.553934 1047 tablet_bootstrap.cc:492] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:41.554165 4409 raft_consensus.cc:2804] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.554500 4409 raft_consensus.cc:493] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.554809 4409 raft_consensus.cc:3060] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:53:41.556551 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 04aff252a46447f49c59345035374580 consensus state has no leader
I20260810 07:53:41.561465 1047 tablet_bootstrap.cc:654] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.567219 1064 ts_tablet_manager.cc:1434] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260810 07:53:41.574128 4409 raft_consensus.cc:515] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.577514 4409 leader_election.cc:290] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.584987 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e15de89beda46f29a650f93f01d4fda" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:41.585510 1064 tablet_bootstrap.cc:492] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:41.585599 895 raft_consensus.cc:3060] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.592509 4217 raft_consensus.cc:493] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.593036 4217 raft_consensus.cc:515] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:41.594655 895 raft_consensus.cc:2468] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:41.595038 4217 leader_election.cc:290] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.595963 4217 raft_consensus.cc:493] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.596660 4389 raft_consensus.cc:493] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.596841 4399 raft_consensus.cc:493] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.597229 4389 raft_consensus.cc:515] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.599287 4389 leader_election.cc:290] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.596889 4217 raft_consensus.cc:515] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.597301 4399 raft_consensus.cc:515] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.601219 4399 leader_election.cc:290] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.602005 4399 raft_consensus.cc:493] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.602492 4399 raft_consensus.cc:515] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.604308 4399 leader_election.cc:290] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.616093 4217 leader_election.cc:290] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.588558 4462 raft_consensus.cc:493] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:41.605346 574 leader_election.cc:304] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.606030 676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83b68614cd8444a08c7820a33c59442d" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.606088 670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0606ccac5d04aa89d804cf9168018b8" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.606774 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2e7619a25b437c8ee2c817255ca638" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.618512 4462 raft_consensus.cc:515] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.619032 4466 raft_consensus.cc:2804] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:53:41.619513 4466 raft_consensus.cc:697] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [term 1 LEADER]: Becoming Leader. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Running, Role: LEADER
I20260810 07:53:41.620599 4462 leader_election.cc:290] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.620311 4466 consensus_queue.cc:237] T 0e15de89beda46f29a650f93f01d4fda P b5bfa873accc428ab48ac19e1207f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.620782 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f7b2b4d5484e08a5e0f56c7e01f07e" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.634397 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1fa9200140a41dfbd886442ae336988" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.635911 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ff256b314e479d981333d9cbfc1656" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:41.636518 895 raft_consensus.cc:2468] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 0.
I20260810 07:53:41.637595 574 leader_election.cc:304] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb, b5bfa873accc428ab48ac19e1207f88d; no voters:
I20260810 07:53:41.638271 4462 raft_consensus.cc:2804] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:53:41.638599 4462 raft_consensus.cc:493] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:41.638907 4462 raft_consensus.cc:3060] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.646307 4462 raft_consensus.cc:515] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.646723 4254 raft_consensus.cc:493] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.647258 4254 raft_consensus.cc:515] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:41.649163 4254 leader_election.cc:290] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917)
I20260810 07:53:41.655860 1047 tablet_bootstrap.cc:492] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d: No bootstrap required, opened a new log
I20260810 07:53:41.656298 1047 ts_tablet_manager.cc:1403] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d: Time spent bootstrapping tablet: real 0.103s user 0.010s sys 0.007s
I20260810 07:53:41.658617 1047 raft_consensus.cc:359] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.659242 1047 raft_consensus.cc:385] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.659505 1047 raft_consensus.cc:740] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bfa873accc428ab48ac19e1207f88d, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.660135 1064 tablet_bootstrap.cc:654] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.660032 892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d24209ef6b4c919ce12a702a318577" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb" is_pre_election: true
I20260810 07:53:41.660418 4462 leader_election.cc:290] T 44ff256b314e479d981333d9cbfc1656 P b5bfa873accc428ab48ac19e1207f88d [CANDIDATE]: Term 1 election: Requested vote from peers 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67:42917), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.662623 1047 consensus_queue.cc:260] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.661063 895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ff256b314e479d981333d9cbfc1656" candidate_uuid: "b5bfa873accc428ab48ac19e1207f88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e545e0597e24d61b7265abed2ccdcfb"
I20260810 07:53:41.663597 895 raft_consensus.cc:3060] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:53:41.664930 4482 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:41.665444 4482 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:41.668964 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:41.669687 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:41.670292 2737 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 5.
I20260810 07:53:41.673203 4483 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:41.673710 4483 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:41.674320 32565 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, 72fde54f23f34224ac756f11924c9531; no voters: d86331b1fae14fb4b49b99f676a8490a
W20260810 07:53:41.674877 800 connection.cc:573] server connection from 127.31.139.65:59453 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:41.676687 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
W20260810 07:53:41.675026 575 connection.cc:573] server connection from 127.31.139.65:45033 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:53:41.677430 32450 connection.cc:573] client connection to 127.31.139.65:39809 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:53:41.677817 32450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 2 similar messages]
I20260810 07:53:41.678711 1047 ts_tablet_manager.cc:1434] T bf6663cbe7fd43a79265d87c2cc11a62 P b5bfa873accc428ab48ac19e1207f88d: Time spent starting tablet: real 0.022s user 0.001s sys 0.005s
W20260810 07:53:41.681763 792 connection.cc:573] client connection to 127.31.139.65:39809 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:41.684827 4217 raft_consensus.cc:493] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
W20260810 07:53:41.685328 573 connection.cc:573] client connection to 127.31.139.65:39809 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:53:41.685787 573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:41.685482 4217 raft_consensus.cc:515] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
W20260810 07:53:41.684952 792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:41.687419 4488 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260810 07:53:41.687831 4488 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d86331b1fae14fb4b49b99f676a8490a)
I20260810 07:53:41.688179 4488 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 5 FOLLOWER]: Advancing to term 6
I20260810 07:53:41.688468 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
I20260810 07:53:41.689621 4482 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:41.688226 4217 leader_election.cc:290] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:41.691614 574 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260810 07:53:41.695951 4483 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:41.707003 4488 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:41.710454 32601 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72fde54f23f34224ac756f11924c9531 in current term 6: Already voted for candidate 2c43f834fbcb447587880136f69c7993 in this term.
I20260810 07:53:41.713765 2566 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:41.715006 4483 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 5 FOLLOWER]: Advancing to term 6
I20260810 07:53:41.721491 1047 tablet_bootstrap.cc:492] T 4fb366297938415382cf6555ed052956 P b5bfa873accc428ab48ac19e1207f88d: Bootstrap starting.
I20260810 07:53:41.723824 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:53:41.724759 671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" candidate_uuid: "9e545e0597e24d61b7265abed2ccdcfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5bfa873accc428ab48ac19e1207f88d" is_pre_election: true
I20260810 07:53:41.727941 4488 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:41.728065 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 6 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:53:41.729501 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c43f834fbcb447587880136f69c7993; no voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:41.730839 895 raft_consensus.cc:2468] T 44ff256b314e479d981333d9cbfc1656 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bfa873accc428ab48ac19e1207f88d in term 1.
I20260810 07:53:41.733619 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 511
I20260810 07:53:41.741382 4488 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260810 07:53:41.743777 4483 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260810 07:53:41.766583 4494 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:53:41.745536 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.31.139.65:39809 (local address 127.0.0.1:50108)
0810 07:53:41.752758 (+ 7222us) negotiation.cc:107] Waiting for socket to connect
0810 07:53:41.753019 (+ 261us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":6445,"threads_started":1}
I20260810 07:53:41.773551 1064 tablet_bootstrap.cc:492] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:41.773406 4217 raft_consensus.cc:493] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.773947 4217 raft_consensus.cc:515] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.775938 4217 leader_election.cc:290] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.776558 1064 ts_tablet_manager.cc:1403] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.191s user 0.005s sys 0.007s
I20260810 07:53:41.779213 1064 raft_consensus.cc:359] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.779857 1064 raft_consensus.cc:385] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.780128 1064 raft_consensus.cc:740] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.787218 32601 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Rejecting Update request from peer d86331b1fae14fb4b49b99f676a8490a for earlier term 5. Current term is 6. Ops: []
I20260810 07:53:41.789763 4484 consensus_queue.cc:1059] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 }, Status: INVALID_TERM, Last received: 5.421, Next index: 422, Last known committed idx: 421, Time since last communication: 0.001s
W20260810 07:53:41.793071 32450 catalog_manager.cc:4777] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): AlterTable RPC failed for tablet f8415b6376a740388487f7ae4aebea52: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection reset by peer (error 104)
I20260810 07:53:41.785209 4217 raft_consensus.cc:493] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.806313 4217 raft_consensus.cc:515] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:41.808337 4217 leader_election.cc:290] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:41.788158 4399 raft_consensus.cc:493] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.809211 4399 raft_consensus.cc:515] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:41.810967 4399 leader_election.cc:290] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:41.797581 792 consensus_peers.cc:597] T 9b4339a26ad540f4b385f69fd0dcdc5d P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:41.790731 4389 raft_consensus.cc:493] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:41.811921 4389 raft_consensus.cc:515] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:41.813968 4389 leader_election.cc:290] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:41.780704 1064 consensus_queue.cc:260] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.816215 4496 raft_consensus.cc:3055] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 LEADER]: Stepping down as leader of term 5
I20260810 07:53:41.816687 4496 raft_consensus.cc:740] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d86331b1fae14fb4b49b99f676a8490a, State: Running, Role: LEADER
W20260810 07:53:41.811941 792 consensus_peers.cc:597] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.817210 792 leader_election.cc:336] T 433404aa29574d819371fe090e670095 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.817854 792 consensus_peers.cc:597] T f65480f06a3c4b2886aa2485f81e16f2 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.818230 792 consensus_peers.cc:597] T 1f8941c49c0a4fba9f7de2aabb790901 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.818686 792 leader_election.cc:336] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.819025 792 leader_election.cc:304] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.819630 4217 raft_consensus.cc:2749] T bdc0039db22d4812a1412e10175d2237 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:41.819844 1064 ts_tablet_manager.cc:1434] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.043s user 0.005s sys 0.000s
W20260810 07:53:41.820447 792 consensus_peers.cc:597] T 4d6df1791e5146a38f3cf0304b5034cb P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.820894 792 leader_election.cc:336] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.821244 792 leader_election.cc:304] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.821861 4217 raft_consensus.cc:2749] T a4863f3f03104d31ad4dafda6f33c9af P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.822654 792 leader_election.cc:336] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.822984 792 leader_election.cc:304] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.823540 4217 raft_consensus.cc:2749] T 8a95df8c34ad4ff4bfe70cea1e0ebb92 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.826480 792 leader_election.cc:336] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.826834 792 leader_election.cc:304] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:41.827605 792 leader_election.cc:336] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.827975 792 leader_election.cc:304] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.828106 4389 raft_consensus.cc:2749] T b7bbccbecf884768b3b171d9bd901b9b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.828523 792 raft_consensus.cc:2686] T afbac73bbeb44af6b39ab23c5cf79eed P 9e545e0597e24d61b7265abed2ccdcfb: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260810 07:53:41.830143 792 consensus_peers.cc:597] T 3b80b61b2e2044eaba3b99e8b0e86f9d P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.830564 792 consensus_peers.cc:597] T b305a232550e4d4babb19093ea1163f0 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.830984 792 consensus_peers.cc:597] T aed14fa26b9a46a4b3e9a43bac05a795 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.831329 792 leader_election.cc:336] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.831696 792 leader_election.cc:304] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.817847 4496 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 421, Committed index: 421, Last appended: 5.421, Last appended by leader: 421, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:41.834748 4217 raft_consensus.cc:2749] T c6080c8b12014fe09d48e307d394f9c9 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:41.843587 4496 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 5 FOLLOWER]: Advancing to term 6
W20260810 07:53:41.832662 792 consensus_peers.cc:597] T c1826ebc26f04805b70ac5314e48fda9 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.845185 792 consensus_peers.cc:597] T ed74e441397344e09e924fb724ad55d6 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.845566 792 consensus_peers.cc:597] T 10341e269a934612bb0cf6ef29818a39 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.845929 792 leader_election.cc:336] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.846276 792 leader_election.cc:304] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
W20260810 07:53:41.846846 792 raft_consensus.cc:2686] T 2905f0597c07439989a00a213fc2194f P 9e545e0597e24d61b7265abed2ccdcfb: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260810 07:53:41.848264 792 consensus_peers.cc:597] T 5eb896c8922e412c97e2cc4891298830 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:41.848378 1064 tablet_bootstrap.cc:492] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
W20260810 07:53:41.848629 792 leader_election.cc:336] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.849005 792 leader_election.cc:304] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.854691 1064 tablet_bootstrap.cc:654] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:41.856920 4389 raft_consensus.cc:2749] T 92c95cd95c034d9a9205221d07ae8615 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.857980 792 leader_election.cc:336] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.858371 792 leader_election.cc:304] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.859015 4389 raft_consensus.cc:2749] T 8c02aece0af740828f4009706445e60c P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.859997 792 leader_election.cc:336] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.860356 792 leader_election.cc:304] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.860992 4389 raft_consensus.cc:2749] T 2c49060c69f2434db4e9f83f2147d7d1 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.861757 792 leader_election.cc:336] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.862133 792 leader_election.cc:304] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.866577 4389 raft_consensus.cc:2749] T f6b54db926fa4032b6408d3d309de35b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.867514 792 leader_election.cc:336] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.867868 792 leader_election.cc:304] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.880954 4389 raft_consensus.cc:2749] T 7c075fdf1055477db9f2d56324f12a8a P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.883364 792 consensus_peers.cc:597] T 2b5c3bed1f2a42b1bbf18b497555eb7a P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.883759 792 leader_election.cc:336] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.884104 792 leader_election.cc:304] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.884704 4217 raft_consensus.cc:2749] T 666f36dafa92421cae47c94593ed8899 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.885655 792 leader_election.cc:336] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.886257 792 leader_election.cc:336] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.886593 792 leader_election.cc:304] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.887207 4217 raft_consensus.cc:2749] T 1e629e89eb734ade82b3dcbf11c971f8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.888018 792 leader_election.cc:336] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.888357 792 leader_election.cc:304] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.888993 4399 raft_consensus.cc:2749] T 4bb34e5b988b497c8f72a6c7266c0f2d P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.889189 792 consensus_peers.cc:597] T 94ced893d6804aa395b73f1f5ed085f9 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.889597 792 leader_election.cc:336] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.889961 792 leader_election.cc:304] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.890578 4399 raft_consensus.cc:2749] T 5a892c4bc5694118b514cc1baa56601d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.890739 792 leader_election.cc:336] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.891361 792 consensus_peers.cc:597] T b705e1630f124370af386436f325004b P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.891731 792 leader_election.cc:336] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.892059 792 leader_election.cc:304] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.892678 4399 raft_consensus.cc:2749] T a9f5a4321a504ed59e0e2ba21ac801f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.892901 792 leader_election.cc:336] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.893261 792 leader_election.cc:304] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.893923 4399 raft_consensus.cc:2749] T 0e15de89beda46f29a650f93f01d4fda P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.894196 792 consensus_peers.cc:597] T 93f62494bb034cd0b4f34f61652569c6 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.894631 792 consensus_peers.cc:597] T e7529f4ac9fb457ba1b3c3bc90eb275d P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.895000 792 leader_election.cc:336] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.895596 792 leader_election.cc:336] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.895955 792 leader_election.cc:304] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.896898 4399 raft_consensus.cc:2749] T 8d92e0a84839427099a9da06e2b970ce P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:41.911984 1064 tablet_bootstrap.cc:492] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:41.919679 1064 ts_tablet_manager.cc:1403] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.003s
I20260810 07:53:41.922273 1064 raft_consensus.cc:359] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:41.922923 1064 raft_consensus.cc:385] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:41.923187 1064 raft_consensus.cc:740] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:41.923792 1064 consensus_queue.cc:260] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
W20260810 07:53:41.896699 792 leader_election.cc:336] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.925112 792 leader_election.cc:304] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.925737 4389 raft_consensus.cc:2749] T bf6663cbe7fd43a79265d87c2cc11a62 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.925948 792 leader_election.cc:336] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.926554 792 consensus_peers.cc:597] T b2fb91a223154dde8ecc2bebf2c0fff9 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.926923 792 leader_election.cc:336] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.927294 792 leader_election.cc:304] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.927991 4389 raft_consensus.cc:2749] T b2cc9ff5b3344dc39e2e560d63f0ce92 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.928076 792 leader_election.cc:336] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.928664 792 leader_election.cc:336] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.929028 792 leader_election.cc:304] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.929666 4389 raft_consensus.cc:2749] T 4fb366297938415382cf6555ed052956 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.929850 792 consensus_peers.cc:597] T e38ab80cd5c441f3ba20bf00577b223c P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.930254 792 leader_election.cc:336] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.930576 792 leader_election.cc:304] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.931213 4389 raft_consensus.cc:2749] T dd79131856f54395a3c5fb551961d4f7 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:53:41.941797 1064 ts_tablet_manager.cc:1434] T a7e077b6b69044c2b6d3d2bb7d99ee29 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260810 07:53:41.943706 1064 tablet_bootstrap.cc:492] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:41.965127 1064 tablet_bootstrap.cc:654] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:41.966689 792 consensus_peers.cc:597] T 91c2e1bcd13a4c22b2141949b2d73188 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:41.967171 792 leader_election.cc:336] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.967528 792 leader_election.cc:304] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.968274 4399 raft_consensus.cc:2749] T a86401e669fa4358ae33ce2249b3a98e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.969195 792 leader_election.cc:336] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.969601 792 leader_election.cc:304] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.970360 4399 raft_consensus.cc:2749] T e20d46d62cdc48669d7474fae00660d7 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:53:41.971201 792 leader_election.cc:336] T 0e5a69fed75047f18522bfee894c769b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:41.971855 792 leader_election.cc:336] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.972252 792 leader_election.cc:304] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.980948 4399 raft_consensus.cc:2749] T a4d1c8d270f744ee9849475c9128414b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.981937 792 leader_election.cc:336] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.982337 792 leader_election.cc:304] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.983861 4389 raft_consensus.cc:2749] T e0a74b22a6864180956e763d2b6b6de6 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.983130 792 leader_election.cc:336] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.993165 792 leader_election.cc:304] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.993938 4399 raft_consensus.cc:2749] T 07fe1a943230432dbe533e7b3671172b P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.994824 792 leader_election.cc:336] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.995221 792 leader_election.cc:304] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.995918 4399 raft_consensus.cc:2749] T 999f3e79ee4b4c368b60b202735c2556 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.996943 792 leader_election.cc:336] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:41.997330 792 leader_election.cc:304] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:41.998733 4399 raft_consensus.cc:2749] T f0606ccac5d04aa89d804cf9168018b8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:41.999732 792 leader_election.cc:336] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.000090 792 leader_election.cc:304] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:42.002180 1064 tablet_bootstrap.cc:492] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:42.002614 1064 ts_tablet_manager.cc:1403] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.003s
I20260810 07:53:42.004311 4399 raft_consensus.cc:2749] T ab2e7619a25b437c8ee2c817255ca638 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:42.005728 1064 raft_consensus.cc:359] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.006348 1064 raft_consensus.cc:385] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:42.006587 1064 raft_consensus.cc:740] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:42.007364 792 leader_election.cc:336] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.007751 792 leader_election.cc:304] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:42.012696 1064 consensus_queue.cc:260] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.013640 4399 raft_consensus.cc:2749] T e1fa9200140a41dfbd886442ae336988 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:42.014577 792 leader_election.cc:336] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.014943 792 leader_election.cc:304] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:42.018771 4399 raft_consensus.cc:2749] T 83b68614cd8444a08c7820a33c59442d P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:53:42.028757 792 leader_election.cc:336] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.029212 792 leader_election.cc:304] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:42.022594 1064 ts_tablet_manager.cc:1434] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260810 07:53:42.030474 1064 tablet_bootstrap.cc:492] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:42.036404 4217 raft_consensus.cc:2749] T 85f7b2b4d5484e08a5e0f56c7e01f07e P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:42.037945 4217 raft_consensus.cc:493] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.038412 4217 raft_consensus.cc:515] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:42.040274 4217 leader_election.cc:290] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.040215 4399 raft_consensus.cc:493] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.040808 4399 raft_consensus.cc:515] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:42.042506 4399 leader_election.cc:290] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.046186 4217 raft_consensus.cc:493] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.046478 4399 raft_consensus.cc:493] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.046672 4217 raft_consensus.cc:515] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.046962 4399 raft_consensus.cc:515] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.048705 4217 leader_election.cc:290] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.048768 4399 leader_election.cc:290] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.067159 4399 raft_consensus.cc:493] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.067680 4399 raft_consensus.cc:515] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.069964 4399 leader_election.cc:290] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.070528 4399 raft_consensus.cc:493] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.070866 1064 tablet_bootstrap.cc:654] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:42.070930 4399 raft_consensus.cc:515] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.072959 4399 leader_election.cc:290] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.076920 4217 raft_consensus.cc:493] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.077421 4217 raft_consensus.cc:515] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.079278 4217 leader_election.cc:290] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.079725 4217 raft_consensus.cc:493] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.080183 4217 raft_consensus.cc:515] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.082145 4217 leader_election.cc:290] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.083216 4217 raft_consensus.cc:493] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.083683 4217 raft_consensus.cc:515] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:42.087078 4217 leader_election.cc:290] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
W20260810 07:53:42.090826 792 leader_election.cc:336] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.091312 792 leader_election.cc:304] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e545e0597e24d61b7265abed2ccdcfb; no voters: 60781ef3c8b84a8788f43d77e1bb3b19, b5bfa873accc428ab48ac19e1207f88d
I20260810 07:53:42.092764 4217 raft_consensus.cc:2749] T b6cb01ce3d97454d9d1e7735e51a76c4 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:53:42.106067 1064 tablet_bootstrap.cc:492] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:42.106580 1064 ts_tablet_manager.cc:1403] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.000s
I20260810 07:53:42.109144 1064 raft_consensus.cc:359] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.109784 1064 raft_consensus.cc:385] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:42.110056 1064 raft_consensus.cc:740] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:42.110693 1064 consensus_queue.cc:260] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.124187 1064 ts_tablet_manager.cc:1434] T 63812f4779914930b4e22d020a51a33b P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260810 07:53:42.125146 1064 tablet_bootstrap.cc:492] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:42.127750 4217 raft_consensus.cc:493] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.128290 4217 raft_consensus.cc:515] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } }
I20260810 07:53:42.130219 4217 leader_election.cc:290] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.130553 32463 catalog_manager.cc:5719] T a86401e669fa4358ae33ce2249b3a98e P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
W20260810 07:53:42.132761 792 leader_election.cc:336] T a61dbafb4f7444e690cea0d003d38f22 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.133193 792 leader_election.cc:336] T d3077b3fe04e406a89560229b76d8242 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.133553 792 leader_election.cc:336] T 01d24209ef6b4c919ce12a702a318577 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.133908 792 leader_election.cc:336] T c43aeafc19a240e6991b69249b0516fa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.134451 792 consensus_peers.cc:597] T 461d70a71f2c44228f4220e7f5fb9c64 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260810 07:53:42.134928 792 leader_election.cc:336] T 82edc39a864f49f8be60e4b36ce3d59c P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.148105 1064 tablet_bootstrap.cc:654] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
W20260810 07:53:42.149364 792 leader_election.cc:336] T 45a3fe9686e0445187c77cf41c563c01 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.149837 792 leader_election.cc:336] T 8718f998623e4f1c8b7b8be815e184a3 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.150235 792 leader_election.cc:336] T 5dad896bfa0e4c5cbc4907a75082b064 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.131896 32463 catalog_manager.cc:5719] T 1cb66183211f45d4a756f70c5e331f87 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.171033 32463 catalog_manager.cc:5719] T 6d67fef331994ca48f1dfce1e3b039f4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } }
W20260810 07:53:42.182440 792 leader_election.cc:336] T 7d6c33f86da645e9bf50e45bc229d0a4 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.182917 792 leader_election.cc:336] T 741404076635419bbbd6ce075fc44ccb P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.183306 792 leader_election.cc:336] T 6db16797cabc48fe8ec82355ce3f16be P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.183663 792 leader_election.cc:336] T e6800738970d47dd8e7cba68945042aa P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.184036 792 leader_election.cc:336] T 5d7024a39aa040c2b00196f668b776e8 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.190491 4217 raft_consensus.cc:493] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.190964 4217 raft_consensus.cc:515] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.193028 4217 leader_election.cc:290] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.193730 4217 raft_consensus.cc:493] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.194190 4217 raft_consensus.cc:515] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.193882 32463 catalog_manager.cc:5719] T ed74e441397344e09e924fb724ad55d6 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.195287 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet ed74e441397344e09e924fb724ad55d6 (table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]). Expected version 1 got 0
I20260810 07:53:42.196163 4217 leader_election.cc:290] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
W20260810 07:53:42.197379 792 leader_election.cc:336] T 55aaf296555b43f9a27f2926054c8d4b P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.221040 792 consensus_peers.cc:597] T 05ef1be8eaa84e6bb5586e68268799a0 P 9e545e0597e24d61b7265abed2ccdcfb -> Peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Couldn't send request to peer 60781ef3c8b84a8788f43d77e1bb3b19. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:42.196208 32463 catalog_manager.cc:5719] T f4a46a037cae4f599398b289bb6a2d84 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.226583 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f4a46a037cae4f599398b289bb6a2d84 (table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c]). Expected version 1 got 0
I20260810 07:53:42.227613 1064 tablet_bootstrap.cc:492] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:42.228084 1064 ts_tablet_manager.cc:1403] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.103s user 0.007s sys 0.008s
I20260810 07:53:42.228756 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 5a0f23d5212a46cc8f731d9c08019858 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]). Expected version 1 got 0
I20260810 07:53:42.230587 1064 raft_consensus.cc:359] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.231217 1064 raft_consensus.cc:385] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:42.231500 1064 raft_consensus.cc:740] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
W20260810 07:53:42.233928 792 leader_election.cc:336] T 2abec82a68a54082afa1d8e72aba6617 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
W20260810 07:53:42.234395 792 leader_election.cc:336] T 74d80c441b2b418994b46250a5c7af44 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.235270 1064 consensus_queue.cc:260] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.233371 32463 catalog_manager.cc:5719] T 23bd442cbd9542ab8a521ededc6b1e90 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.244628 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]). Expected version 2 got 0
I20260810 07:53:42.251487 1064 ts_tablet_manager.cc:1434] T 87ae23cb445e43fc9dc2f45097190fec P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.023s user 0.003s sys 0.005s
I20260810 07:53:42.252435 1064 tablet_bootstrap.cc:492] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:42.266265 32463 catalog_manager.cc:5719] T 999f3e79ee4b4c368b60b202735c2556 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.271747 1064 tablet_bootstrap.cc:654] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:42.278952 32463 catalog_manager.cc:5719] T 92c95cd95c034d9a9205221d07ae8615 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.293769 32463 catalog_manager.cc:5719] T 5eb896c8922e412c97e2cc4891298830 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.296795 32463 catalog_manager.cc:5719] T 69f1405e9526433a8fd727ed237f8c84 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.305594 4509 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.306161 4509 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:42.309464 32463 catalog_manager.cc:5719] T ebfc73a83a0f4048ad9dded62d074e19 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.311051 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:42.310860 32463 catalog_manager.cc:5719] T 2a91af48fe6a4bd986b5266f09fad6b2 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.311592 32483 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fde54f23f34224ac756f11924c9531 in term 6.
I20260810 07:53:42.313097 2560 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72fde54f23f34224ac756f11924c9531, d86331b1fae14fb4b49b99f676a8490a; no voters:
I20260810 07:53:42.314936 1064 tablet_bootstrap.cc:492] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:42.315333 1064 ts_tablet_manager.cc:1403] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.063s user 0.012s sys 0.004s
I20260810 07:53:42.325795 4217 raft_consensus.cc:493] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.326545 4217 raft_consensus.cc:515] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.329841 4217 leader_election.cc:290] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.334563 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993" is_pre_election: true
W20260810 07:53:42.335119 792 leader_election.cc:336] T 140522b9e6154f7daca10fdf0c48e6e2 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.335168 32601 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fde54f23f34224ac756f11924c9531 in term 6.
I20260810 07:53:42.336869 4509 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:42.337107 1064 raft_consensus.cc:359] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.338003 1064 raft_consensus.cc:385] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:42.338281 1064 raft_consensus.cc:740] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:42.337858 4509 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260810 07:53:42.341602 4509 raft_consensus.cc:493] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:42.341492 1064 consensus_queue.cc:260] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.342509 4509 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 6 FOLLOWER]: Advancing to term 7
I20260810 07:53:42.349488 4514 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.312548 32463 catalog_manager.cc:5719] T 1e629e89eb734ade82b3dcbf11c971f8 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.352295 1064 ts_tablet_manager.cc:1434] T cca378598d0c45178e497fd212638d77 P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.037s user 0.002s sys 0.005s
I20260810 07:53:42.353323 1064 tablet_bootstrap.cc:492] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:42.359236 4509 raft_consensus.cc:515] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:42.360217 4514 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:42.362494 4514 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:42.364923 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a" is_pre_election: true
I20260810 07:53:42.365516 32483 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 6.
I20260810 07:53:42.361217 32463 catalog_manager.cc:5719] T 0d3aaf0099934d1e9bede4687b36ce44 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.366840 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters:
I20260810 07:53:42.369474 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 461d70a71f2c44228f4220e7f5fb9c64 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279]). Expected version 2 got 0
I20260810 07:53:42.370028 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet cca378598d0c45178e497fd212638d77 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]). Expected version 1 got 0
I20260810 07:53:42.370707 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f8415b6376a740388487f7ae4aebea52 (table default.Table_fe0f90fb5d4445a0abb442761bb12227 [id=0ab295f37b5e482482053f67fe0719a6]). Expected version 1 got 0
I20260810 07:53:42.371326 4514 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260810 07:53:42.371282 32463 catalog_manager.cc:5719] T 5d7024a39aa040c2b00196f668b776e8 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:42.372278 4514 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:53:42.373046 4514 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 6 FOLLOWER]: Advancing to term 7
I20260810 07:53:42.363380 2737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531" is_pre_election: true
I20260810 07:53:42.389875 4514 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:42.372560 32463 catalog_manager.cc:5719] T 433404aa29574d819371fe090e670095 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } }
I20260810 07:53:42.393219 2737 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c43f834fbcb447587880136f69c7993 in current term 7: Already voted for candidate 72fde54f23f34224ac756f11924c9531 in this term.
I20260810 07:53:42.393141 2741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "72fde54f23f34224ac756f11924c9531"
I20260810 07:53:42.393057 32463 catalog_manager.cc:5719] T 55aaf296555b43f9a27f2926054c8d4b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 2, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 2 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:42.394225 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]). Expected version 1 got 0
I20260810 07:53:42.396955 32483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c43f834fbcb447587880136f69c7993" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:53:42.397498 32483 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 6 FOLLOWER]: Advancing to term 7
I20260810 07:53:42.404809 1064 tablet_bootstrap.cc:654] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:42.406832 32471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "d86331b1fae14fb4b49b99f676a8490a"
I20260810 07:53:42.407151 32601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72fde54f23f34224ac756f11924c9531" candidate_term: 7 candidate_status { last_received { term: 5 index: 421 } } ignore_live_leader: false dest_uuid: "2c43f834fbcb447587880136f69c7993"
I20260810 07:53:42.407831 32601 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 72fde54f23f34224ac756f11924c9531 in current term 7: Already voted for candidate 2c43f834fbcb447587880136f69c7993 in this term.
I20260810 07:53:42.408903 32483 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c43f834fbcb447587880136f69c7993 in term 7.
I20260810 07:53:42.409145 2566 leader_election.cc:304] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72fde54f23f34224ac756f11924c9531; no voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a
I20260810 07:53:42.410102 4513 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260810 07:53:42.411128 32563 leader_election.cc:304] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c43f834fbcb447587880136f69c7993, d86331b1fae14fb4b49b99f676a8490a; no voters: 72fde54f23f34224ac756f11924c9531
I20260810 07:53:42.411998 4515 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 7 FOLLOWER]: Leader election won for term 7
I20260810 07:53:42.410115 4509 leader_election.cc:290] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [CANDIDATE]: Term 7 election: Requested vote from peers d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845), 2c43f834fbcb447587880136f69c7993 (127.31.139.124:36273)
I20260810 07:53:42.419374 4514 leader_election.cc:290] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [CANDIDATE]: Term 7 election: Requested vote from peers 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193), d86331b1fae14fb4b49b99f676a8490a (127.31.139.125:46845)
I20260810 07:53:42.424780 4515 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [term 7 LEADER]: Becoming Leader. State: Replica: 2c43f834fbcb447587880136f69c7993, State: Running, Role: LEADER
I20260810 07:53:42.425766 4515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 421, Committed index: 421, Last appended: 5.421, Last appended by leader: 421, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } }
I20260810 07:53:42.443993 32463 catalog_manager.cc:5719] T 83b68614cd8444a08c7820a33c59442d P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.445226 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c43f834fbcb447587880136f69c7993. Latest consensus state: current_term: 7 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:42.445958 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: LEADER
I20260810 07:53:42.445444 32463 catalog_manager.cc:5719] T 0d7084712ff04ac48a54171534e04d69 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.446996 32463 catalog_manager.cc:5719] T 7d6c33f86da645e9bf50e45bc229d0a4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.448019 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017]). Expected version 1 got 0
I20260810 07:53:42.450426 4389 raft_consensus.cc:493] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:53:42.450982 4389 raft_consensus.cc:515] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.452929 4389 leader_election.cc:290] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655), 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809)
I20260810 07:53:42.455940 4522 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260810 07:53:42.458585 792 leader_election.cc:336] T 3dfa1b6f604c452e9f346ecc18b5b691 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.461937 1064 tablet_bootstrap.cc:492] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb: No bootstrap required, opened a new log
I20260810 07:53:42.462448 1064 ts_tablet_manager.cc:1403] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.008s
I20260810 07:53:42.472278 1064 raft_consensus.cc:359] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.473012 1064 raft_consensus.cc:385] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:53:42.473279 1064 raft_consensus.cc:740] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e545e0597e24d61b7265abed2ccdcfb, State: Initialized, Role: FOLLOWER
I20260810 07:53:42.473946 1064 consensus_queue.cc:260] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } }
I20260810 07:53:42.475488 1064 ts_tablet_manager.cc:1434] T 446ed19d914548ea864a43b2bb8c7f5a P 9e545e0597e24d61b7265abed2ccdcfb: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260810 07:53:42.496068 32463 catalog_manager.cc:5719] T a3642f29b38241be9db3cf37ccde2039 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.499449 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83]
I20260810 07:53:42.501853 1064 tablet_bootstrap.cc:492] T 4ecc6870602043269daa61cb173108b3 P 9e545e0597e24d61b7265abed2ccdcfb: Bootstrap starting.
I20260810 07:53:42.504038 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72]
I20260810 07:53:42.505719 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97]
I20260810 07:53:42.507411 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250]
I20260810 07:53:42.507500 1064 tablet_bootstrap.cc:654] T 4ecc6870602043269daa61cb173108b3 P 9e545e0597e24d61b7265abed2ccdcfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:53:42.508976 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_332f1a619c404512a115b8f6cb52402f [id=0548157d3d3c49cca88607a747dfd017]
I20260810 07:53:42.510607 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19]
I20260810 07:53:42.510697 32463 catalog_manager.cc:5719] T e2fe3153512b40a99caf3fb9b81fd7a1 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.512040 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]). Expected version 2 got 0
I20260810 07:53:42.512200 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432]
I20260810 07:53:42.512934 32463 catalog_manager.cc:5719] T 3c97418290e04f60884c721ded14495a P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.513967 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_fe0f90fb5d4445a0abb442761bb12227 [id=0ab295f37b5e482482053f67fe0719a6]
I20260810 07:53:42.514299 32463 catalog_manager.cc:5719] T d3077b3fe04e406a89560229b76d8242 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:42.515664 32463 catalog_manager.cc:5719] T 2c49060c69f2434db4e9f83f2147d7d1 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.517158 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c]
I20260810 07:53:42.518774 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6]
I20260810 07:53:42.520246 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5]
I20260810 07:53:42.523396 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]
I20260810 07:53:42.537922 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6]
I20260810 07:53:42.544158 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017]
I20260810 07:53:42.546056 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e]
I20260810 07:53:42.548468 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f]
I20260810 07:53:42.553450 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_14a4faae60bb4b7795842a1172472aeb [id=1d0f4f7a9d0f4230b27d2470fba7a5d4]
I20260810 07:53:42.555104 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995]
I20260810 07:53:42.554481 32463 catalog_manager.cc:5719] T 8ea09585009e4a3ea33f620182bfee60 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.556146 32463 catalog_manager.cc:5719] T 93f62494bb034cd0b4f34f61652569c6 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } }
I20260810 07:53:42.556699 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de]
I20260810 07:53:42.557677 32463 catalog_manager.cc:5719] T c6080c8b12014fe09d48e307d394f9c9 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.559031 32463 catalog_manager.cc:5719] T 1fe71e69dfd74a3b826050bd7edc6b6a P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.560140 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015]). Expected version 1 got 0
W20260810 07:53:42.565155 792 connection.cc:573] client connection to 127.31.139.66:37655 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:42.568863 4389 raft_consensus.cc:493] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60781ef3c8b84a8788f43d77e1bb3b19)
I20260810 07:53:42.560670 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf]
I20260810 07:53:42.569365 4389 raft_consensus.cc:515] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } }
I20260810 07:53:42.571290 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc]
I20260810 07:53:42.571511 4389 leader_election.cc:290] T fc12205a097c496cba5b777b98f27914 P 9e545e0597e24d61b7265abed2ccdcfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809), b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655)
I20260810 07:53:42.573820 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_a676bcc87b0c4673adb4eb1808a05542 [id=2b9cf9a938964ee0bd5f7cdeb5a8cb37]
I20260810 07:53:42.578501 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55]
W20260810 07:53:42.589661 4495 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:53:42.579981 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.31.139.66:37655 (local address 127.31.139.67:34291)
0810 07:53:42.580979 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0810 07:53:42.581024 (+ 45us) client_negotiation.cc:175] Beginning negotiation
0810 07:53:42.581486 (+ 462us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 07:53:42.588829 (+ 7343us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.139.66:37655: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":402,"threads_started":1}
W20260810 07:53:42.588897 32565 connection.cc:573] server connection from 127.31.139.66:48899 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:42.597533 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43]
I20260810 07:53:42.609136 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 756
I20260810 07:53:42.620194 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9]
I20260810 07:53:42.622843 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408]
I20260810 07:53:42.622499 32463 catalog_manager.cc:5719] T 849197942f134542a387a10167fc7481 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.624043 32463 catalog_manager.cc:5719] T 3b80b61b2e2044eaba3b99e8b0e86f9d P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.624655 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27]
I20260810 07:53:42.625358 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426]). Expected version 1 got 0
I20260810 07:53:42.626801 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186]
I20260810 07:53:42.629390 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a]
W20260810 07:53:42.631003 32450 catalog_manager.cc:4832] Async tablet task AlterTable RPC for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 04aff252a46447f49c59345035374580 consensus state has no leader
I20260810 07:53:42.631057 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf]
I20260810 07:53:42.632757 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855]
I20260810 07:53:42.625983 32463 catalog_manager.cc:5719] T a4d1c8d270f744ee9849475c9128414b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.634546 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_1315259f9b4e4425a2f99c5125e402bc [id=3b6a4e9c10744d4dbef52ed4a87fc36f]
I20260810 07:53:42.636163 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f]
I20260810 07:53:42.637794 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1]
I20260810 07:53:42.637436 32463 catalog_manager.cc:5719] T 6db16797cabc48fe8ec82355ce3f16be P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.639549 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70]
I20260810 07:53:42.639072 32463 catalog_manager.cc:5719] T 9350b151227d45e6bae8adddd8fff86b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.640691 32463 catalog_manager.cc:5719] T c47792551bb243b7b3f6de904c17c47b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.641275 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30]
I20260810 07:53:42.641782 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet c47792551bb243b7b3f6de904c17c47b (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]). Expected version 1 got 0
I20260810 07:53:42.642629 32463 catalog_manager.cc:5719] T b2fb91a223154dde8ecc2bebf2c0fff9 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } }
I20260810 07:53:42.643184 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb]
I20260810 07:53:42.643814 32463 catalog_manager.cc:5719] T b705e1630f124370af386436f325004b P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } }
I20260810 07:53:42.645084 32463 catalog_manager.cc:5719] T 140522b9e6154f7daca10fdf0c48e6e2 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } }
I20260810 07:53:42.646328 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f2ff63903b4c44fdbc86a2d11306de7d [id=466e3662813242f9b2c85763c28ae42c]
I20260810 07:53:42.646294 32463 catalog_manager.cc:5719] T cc9959255b4d46e29b9164027638e38e P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.648007 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4]
I20260810 07:53:42.649678 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5]
I20260810 07:53:42.651173 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812]
I20260810 07:53:42.647984 32463 catalog_manager.cc:5719] T aed14fa26b9a46a4b3e9a43bac05a795 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.652796 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]). Expected version 1 got 0
I20260810 07:53:42.652963 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a]
I20260810 07:53:42.653446 32463 catalog_manager.cc:5719] T a61dbafb4f7444e690cea0d003d38f22 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.654487 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet a61dbafb4f7444e690cea0d003d38f22 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]). Expected version 3 got 0
I20260810 07:53:42.654695 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13]
I20260810 07:53:42.655134 32463 catalog_manager.cc:5719] T 6f17bb6c7ad74c178e1b5d96b7e688b7 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } }
I20260810 07:53:42.656358 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_52b389643dbe4064ba2010a1d2d6c8aa [id=55e0c4c593154fd8bb50dcfe3c932719]
I20260810 07:53:42.656713 32463 catalog_manager.cc:5719] T 4d6df1791e5146a38f3cf0304b5034cb P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.657817 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]). Expected version 2 got 0
I20260810 07:53:42.658061 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18]
I20260810 07:53:42.658806 32463 catalog_manager.cc:5719] T 01d24209ef6b4c919ce12a702a318577 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:42.659904 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9]
I20260810 07:53:42.660002 32463 catalog_manager.cc:5719] T 6cd9484084294acfba9613fa49f369b2 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.661139 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet 6cd9484084294acfba9613fa49f369b2 (table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]). Expected version 1 got 0
I20260810 07:53:42.661772 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet e0a74b22a6864180956e763d2b6b6de6 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]). Expected version 1 got 0
I20260810 07:53:42.662550 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866]
I20260810 07:53:42.662706 32463 catalog_manager.cc:5719] T f65480f06a3c4b2886aa2485f81e16f2 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.663796 32463 catalog_manager.cc:5811] TS b5bfa873accc428ab48ac19e1207f88d (127.31.139.66:37655) does not have the latest schema for tablet f65480f06a3c4b2886aa2485f81e16f2 (table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]). Expected version 1 got 0
I20260810 07:53:42.665606 32463 catalog_manager.cc:5719] T e7529f4ac9fb457ba1b3c3bc90eb275d P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e545e0597e24d61b7265abed2ccdcfb (127.31.139.67). New cstate: current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.666226 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb]
I20260810 07:53:42.667038 32463 catalog_manager.cc:5719] T 9f6e61aa223048f8bdc9eee07fa2066c P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bfa873accc428ab48ac19e1207f88d (127.31.139.66). New cstate: current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260810 07:53:42.668040 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1]
I20260810 07:53:42.668497 32463 catalog_manager.cc:5719] T 834a48ebb87a4b999722e05af1feab21 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } }
I20260810 07:53:42.669790 32463 catalog_manager.cc:5719] T b6cb01ce3d97454d9d1e7735e51a76c4 P b5bfa873accc428ab48ac19e1207f88d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } }
I20260810 07:53:42.671641 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015]
I20260810 07:53:42.673266 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9]
I20260810 07:53:42.674772 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133]
I20260810 07:53:42.677125 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614]
I20260810 07:53:42.678748 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f]
I20260810 07:53:42.680429 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898]
I20260810 07:53:42.682027 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0]
I20260810 07:53:42.685326 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23]
I20260810 07:53:42.687105 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45]
I20260810 07:53:42.689018 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1]
I20260810 07:53:42.691184 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4]
I20260810 07:53:42.694087 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d]
I20260810 07:53:42.696097 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d]
I20260810 07:53:42.698982 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb]
I20260810 07:53:42.700712 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e]
I20260810 07:53:42.703307 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_10ebf20223374508adbac3c536614ad3 [id=752ea2a5c0d54b8a837d6b8e7f247d73]
I20260810 07:53:42.704841 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4]
I20260810 07:53:42.706265 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5]
I20260810 07:53:42.707684 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_80bd87f8139444628cd4688972911389 [id=7e4af06cb73544f29f99a692ae24e853]
I20260810 07:53:42.709214 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e]
I20260810 07:53:42.710767 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1]
I20260810 07:53:42.712177 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4]
I20260810 07:53:42.713660 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b]
I20260810 07:53:42.715133 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661]
I20260810 07:53:42.716575 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141]
I20260810 07:53:42.719072 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262]
I20260810 07:53:42.720481 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264]
I20260810 07:53:42.724884 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d]
I20260810 07:53:42.726284 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a]
I20260810 07:53:42.729349 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067]
I20260810 07:53:42.730866 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a]
I20260810 07:53:42.732318 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f]
E20260810 07:53:42.686056 32463 catalog_manager.cc:5874] Error updating tablets from b5bfa873accc428ab48ac19e1207f88d: Illegal state: replica d86331b1fae14fb4b49b99f676a8490a is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: false updated_tablets { tablet_id: "09e234449dc6407585dc79cfdce766cc" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "fce424ac9b4048c4a988f0fa48acbf46" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "505e2f7a25a447c3bf19b310aca1702c" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "38f8faef6d284c98bf5fd9f7c6f9c443" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "63812f4779914930b4e22d020a51a33b" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "446ed19d914548ea864a43b2bb8c7f5a" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "3dfa1b6f604c452e9f346ecc18b5b691" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "bf6663cbe7fd43a79265d87c2cc11a62" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "0e5a69fed75047f18522bfee894c769b" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "44ff256b314e479d981333d9cbfc1656" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "dd79131856f54395a3c5fb551961d4f7" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "05ef1be8eaa84e6bb5586e68268799a0" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "4ecc6870602043269daa61cb173108b3" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "5a892c4bc5694118b514cc1baa56601d" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "a9f5a4321a504ed59e0e2ba21ac801f7" state: RUNNING consensus_state { current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "04aff252a46447f49c59345035374580" state: RUNNING consensus_state { current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "f6cd26a04d3c417d87091f05bc5cb5cd" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "bdc0039db22d4812a1412e10175d2237" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27281 live_row_count: 0 } } updated_tablets { tablet_id: "e0a74b22a6864180956e763d2b6b6de6" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "1d25b4b974034eea9f1256fb4e2a503c" state: RUNNING consensus_state { current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "94ced893d6804aa395b73f1f5ed085f9" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "e7529f4ac9fb457ba1b3c3bc90eb275d" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "87ae23cb445e43fc9dc2f45097190fec" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "91c2e1bcd13a4c22b2141949b2d73188" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "9b4339a26ad540f4b385f69fd0dcdc5d" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "b6cb01ce3d97454d9d1e7735e51a76c4" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "f65480f06a3c4b2886aa2485f81e16f2" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "aed14fa26b9a46a4b3e9a43bac05a795" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "140522b9e6154f7daca10fdf0c48e6e2" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "b705e1630f124370af386436f325004b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "b2fb91a223154dde8ecc2bebf2c0fff9" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "a61dbafb4f7444e690cea0d003d38f22" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "6f17bb6c7ad74c178e1b5d96b7e688b7" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "c47792551bb243b7b3f6de904c17c47b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27356 live_row_count: 0 } } updated_tablets { tablet_id: "6a97561b5ff5490890e756ca5ff50d38" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27356 live_row_count: 0 } } updated_tablets { tablet_id: "6db16797cabc48fe8ec82355ce3f16be" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "9350b151227d45e6bae8adddd8fff86b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27356 live_row_count: 0 } } updated_tablets { tablet_id: "fe8b4892a53742f48dd43397c4f8e88f" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27356 live_row_count: 0 } } updated_tablets { tablet_id: "741404076635419bbbd6ce075fc44ccb" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "74d80c441b2b418994b46250a5c7af44" state: INITIALIZED consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "a6077111ffb2411892b62d806ea3c85b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "a4d1c8d270f744ee9849475c9128414b" state: RUNNING consensus_state { current_term: 1 leader_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } updated_tablets { tablet_id: "849197942f134542a387a10167fc7481" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b5bfa873accc428ab48ac19e1207f88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5bfa873accc428ab48ac19e1207f88d" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 37655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e545e0597e24d61b7265abed2ccdcfb" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 42917 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { on_disk_size: 27356 live_row_count: 0 } } updated_tablets { tablet_id: "3b80b61b2e2044eaba3b99e8b0e86f9d" state: RUNNING consensus_state { current_term: 1 leader_uuid: "9e545e0597e24d61b7265abed2ccdcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60781ef3c8b84a8788f43d77e1bb3b19" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 39809 } } peers
I20260810 07:53:42.734651 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7]
I20260810 07:53:42.738270 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d]
W20260810 07:53:42.739754 32463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 106) took 1362 ms (client timeout 1000 ms). Trace:
I20260810 07:53:42.740525 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793]
W20260810 07:53:42.740103 32463 rpcz_store.cc:269] 0810 07:53:41.377556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:53:41.379675 (+ 2119us) service_pool.cc:224] Handling call
0810 07:53:42.685795 (+1306120us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:42.685818 (+ 23us) write_op.cc:454] Released schema lock
0810 07:53:42.739654 (+ 53836us) inbound_call.cc:177] Queueing failure response
Related trace 'op':
0810 07:53:42.685547 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d86331b1fae14fb4b49b99f676a8490a is not leader of this config: current role FOLLOWER)
0810 07:53:42.685608 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:53:42.685630 (+ 22us) write_op.cc:454] Released schema lock
0810 07:53:42.685652 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":131,"child_traces":[["op",{}]]}
W20260810 07:53:42.742012 32455 connection.cc:466] server connection from 127.31.139.66:41637 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.31.139.66:41637 (request call id 106) could send its response
I20260810 07:53:42.744778 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653]
I20260810 07:53:42.746454 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92]
I20260810 07:53:42.750102 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643]
I20260810 07:53:42.753023 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a]
I20260810 07:53:42.757303 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_110c350dc5474f2fbe9bbd9410aac109 [id=b23efee8dd99419d9308ef2d5a907409]
I20260810 07:53:42.758654 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d]
I20260810 07:53:42.760076 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb]
I20260810 07:53:42.762214 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c]
I20260810 07:53:42.763669 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257]
I20260810 07:53:42.765116 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b]
I20260810 07:53:42.767349 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d]
I20260810 07:53:42.768965 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72]
I20260810 07:53:42.773435 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb]
I20260810 07:53:42.775867 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7]
I20260810 07:53:42.777370 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9]
I20260810 07:53:42.778882 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_7b1b0b8833e640e1bdddb0a030106d84 [id=d807e002aab64400be9e50e1e447255a]
I20260810 07:53:42.781100 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b]
I20260810 07:53:42.783174 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a]
I20260810 07:53:42.785140 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e]
I20260810 07:53:42.786592 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_d8cf507b1d004b598ab0cb354102eec1 [id=e0c72595749b40ac8a72e5312d1e241b]
I20260810 07:53:42.788062 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da]
I20260810 07:53:42.789529 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24]
I20260810 07:53:42.792799 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079]
I20260810 07:53:42.794260 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5]
I20260810 07:53:42.795779 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446]
I20260810 07:53:42.797222 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e]
I20260810 07:53:42.798636 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961]
I20260810 07:53:42.800091 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f]
W20260810 07:53:42.802228 32450 catalog_manager.cc:4777] TS 60781ef3c8b84a8788f43d77e1bb3b19 (127.31.139.65:39809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 705fc015928140a58db979445068365f: Network error: Client connection negotiation failed: client connection to 127.31.139.65:39809: connect: Connection refused (error 111)
I20260810 07:53:42.803046 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8]
I20260810 07:53:42.805073 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_e972e7a884334e7aaf450c0be103a827 [id=f26d526a9f1949df98cb9ff907fbfe73]
I20260810 07:53:42.806797 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279]
I20260810 07:53:42.808475 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957]
I20260810 07:53:42.810940 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_3a6b61ca82b74657a1283940e4704e5e [id=f6964bf688dd4cfcb736b23eb6108219]
I20260810 07:53:42.812564 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca]
I20260810 07:53:42.814141 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168]
I20260810 07:53:42.815527 4522 catalog_manager.cc:679] Loaded metadata for table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151]
I20260810 07:53:42.843693 4522 tablet_loader.cc:96] loaded metadata for tablet 01d24209ef6b4c919ce12a702a318577 (table default.Table_11aa8b64aab44d1fb8a173833ac51e11 [id=5d7a64720d474e9e8b54d46a980928f1])
I20260810 07:53:42.847429 4522 tablet_loader.cc:96] loaded metadata for tablet 04aff252a46447f49c59345035374580 (table default.Table_170c952671b84b36873085f10e575663 [id=837b105c639d460fbd573628e3778262])
I20260810 07:53:42.849145 4522 tablet_loader.cc:96] loaded metadata for tablet 05ef1be8eaa84e6bb5586e68268799a0 (table default.Table_e5631a682bd14d889ec772c94d4a5c5b [id=dd506fe4b1714eacb34f9067957be59a])
I20260810 07:53:42.850710 4522 tablet_loader.cc:96] loaded metadata for tablet 07fe1a943230432dbe533e7b3671172b (table default.Table_e44a2c3484b947aa9c4953108b45c259 [id=413fb6b94d69441990fb42a0f36c5dbb])
I20260810 07:53:42.855399 2737 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 72fde54f23f34224ac756f11924c9531 [term 7 FOLLOWER]: Refusing update from remote peer 2c43f834fbcb447587880136f69c7993: Log matching property violated. Preceding OpId in replica: term: 5 index: 421. Preceding OpId from leader: term: 7 index: 422. (index mismatch)
I20260810 07:53:42.857388 4514 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 422, Last known committed idx: 421, Time since last communication: 0.001s
I20260810 07:53:42.856330 4522 tablet_loader.cc:96] loaded metadata for tablet 09e234449dc6407585dc79cfdce766cc (table default.Table_7b1b0b8833e640e1bdddb0a030106d84 [id=d807e002aab64400be9e50e1e447255a])
I20260810 07:53:42.868479 4522 tablet_loader.cc:96] loaded metadata for tablet 0d3aaf0099934d1e9bede4687b36ce44 (table default.Table_df5853b3486743289def86718544ef6a [id=99a4c04513ab4dd79975c81b4c77417a])
I20260810 07:53:42.871359 4522 tablet_loader.cc:96] loaded metadata for tablet 0d7084712ff04ac48a54171534e04d69 (table default.Table_2a1f8373fc464a359e73cd83b7381720 [id=d17d541bb07c43f7b56cdcd58137fedb])
I20260810 07:53:42.874500 4522 tablet_loader.cc:96] loaded metadata for tablet 0dbe1a6581374213be7c3f63a7a567a4 (table default.Table_d11ccb67e67e4e2f8c29b4c6cdecd2d9 [id=32f4818f2b09496d8cfddf1516c84186])
I20260810 07:53:42.879731 32483 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [term 7 FOLLOWER]: Refusing update from remote peer 2c43f834fbcb447587880136f69c7993: Log matching property violated. Preceding OpId in replica: term: 5 index: 421. Preceding OpId from leader: term: 7 index: 422. (index mismatch)
W20260810 07:53:42.884856 2566 connection.cc:573] server connection from 127.31.139.67:50457 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:53:42.882022 4514 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 422, Last known committed idx: 421, Time since last communication: 0.000s
I20260810 07:53:42.896116 4522 tablet_loader.cc:96] loaded metadata for tablet 0e15de89beda46f29a650f93f01d4fda (table default.Table_feb396bb0c8345699ece4d8e1245375b [id=34656bcbd0104f1fb8053d093f703a1a])
I20260810 07:53:42.898967 4522 tablet_loader.cc:96] loaded metadata for tablet 0e5a69fed75047f18522bfee894c769b (table default.Table_b76493904a5946c7b069e14e1fbdd9af [id=3832d17905b3476f853631caf96f23bf])
I20260810 07:53:42.900238 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 1292
I20260810 07:53:42.908242 4522 tablet_loader.cc:96] loaded metadata for tablet 10341e269a934612bb0cf6ef29818a39 (table default.Table_d446ae528a0144368b59327e55dd86e1 [id=550221c8ac764a77ab51a5652d0c8a13])
I20260810 07:53:42.912117 4522 tablet_loader.cc:96] loaded metadata for tablet 140522b9e6154f7daca10fdf0c48e6e2 (table default.Table_6fb07c2480e74dfc9cd95d724ec163cc [id=ed7a00eabee545abb686c78ebfd8a961])
I20260810 07:53:42.914444 4522 tablet_loader.cc:96] loaded metadata for tablet 1cb66183211f45d4a756f70c5e331f87 (table default.Table_58e123d14b1d4972bfbc136cb2e341e2 [id=7ad8b867240f4717be4c9f7f7bad76d5])
I20260810 07:53:42.914964 4531 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c43f834fbcb447587880136f69c7993. Latest consensus state: current_term: 7 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:42.915727 4522 tablet_loader.cc:96] loaded metadata for tablet 1d25b4b974034eea9f1256fb4e2a503c (table default.Table_1256df14cdbb41e7b60d872ca8670702 [id=9a7bf9d952b945b080672bcc75d4ed3f])
I20260810 07:53:42.916059 4531 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:42.917071 4522 tablet_loader.cc:96] loaded metadata for tablet 1e629e89eb734ade82b3dcbf11c971f8 (table default.Table_0da354d9c37742bc843f199cb929c391 [id=3ff1d3f4cb054551a94b1aaa98e35c30])
I20260810 07:53:42.918300 4522 tablet_loader.cc:96] loaded metadata for tablet 1f8941c49c0a4fba9f7de2aabb790901 (table default.Table_2101337b99f744bebd7fe77c3777e236 [id=0a828cafe0a243939f99737c02d22432])
I20260810 07:53:42.919479 4522 tablet_loader.cc:96] loaded metadata for tablet 1fe71e69dfd74a3b826050bd7edc6b6a (table default.Table_843816de8aff46309a751a007da50fa6 [id=616669a2db4f43d9b352efabdfc7b015])
I20260810 07:53:42.922428 4522 tablet_loader.cc:96] loaded metadata for tablet 23bd442cbd9542ab8a521ededc6b1e90 (table default.Table_b804194501bd4439adcce524793449cd [id=7e8c44d3660746d69e0982f1cf2dc80e])
I20260810 07:53:42.929064 4531 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:42.931453 4522 tablet_loader.cc:96] loaded metadata for tablet 2a91af48fe6a4bd986b5266f09fad6b2 (table default.Table_d4ab5ccfe1de44e092fb9a3183698973 [id=b55c7cec73734aefb55370b2b49ddaeb])
I20260810 07:53:42.931566 4531 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86331b1fae14fb4b49b99f676a8490a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:53:42.932762 4522 tablet_loader.cc:96] loaded metadata for tablet 2abec82a68a54082afa1d8e72aba6617 (table default.Table_fee6b63b40ed407dbcf843cd9b93b336 [id=c47e6606b13f479487c6f42050b57e72])
I20260810 07:53:42.932777 4515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "2c43f834fbcb447587880136f69c7993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72fde54f23f34224ac756f11924c9531" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38193 } } peers { permanent_uuid: "d86331b1fae14fb4b49b99f676a8490a" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 46845 } } peers { permanent_uuid: "2c43f834fbcb447587880136f69c7993" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 36273 } } }
I20260810 07:53:42.933629 4515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 [sys.catalog]: This master's current role is: LEADER
I20260810 07:53:42.934080 4522 tablet_loader.cc:96] loaded metadata for tablet 2b5c3bed1f2a42b1bbf18b497555eb7a (table default.Table_3ef130273507459b99d40cdeae4ac31f [id=6ee12b0d86354cd4b24f5d1f50bef5a4])
I20260810 07:53:42.935868 4522 tablet_loader.cc:96] loaded metadata for tablet 2c49060c69f2434db4e9f83f2147d7d1 (table default.Table_e1294c8e0ca349b9b06af3a9b62a88cb [id=9ed87c6d126d4e4292a3c2a323223e3d])
I20260810 07:53:42.937309 4522 tablet_loader.cc:96] loaded metadata for tablet 2d8d439276e5440c8d7082b38a1dd3ed (table default.Table_52b389643dbe4064ba2010a1d2d6c8aa [id=55e0c4c593154fd8bb50dcfe3c932719])
I20260810 07:53:42.939563 4522 tablet_loader.cc:96] loaded metadata for tablet 38f8faef6d284c98bf5fd9f7c6f9c443 (table default.Table_d4eb4954a123470aad7f2107c6680c2c [id=b2babac9bd16405a971f83fddeb5659d])
I20260810 07:53:42.940433 4522 tablet_loader.cc:96] loaded metadata for tablet 3b80b61b2e2044eaba3b99e8b0e86f9d (table default.Table_d83c2642cc224dc69bb060597fbdfc63 [id=830a1c1edaa24998a1a916d8e6279141])
I20260810 07:53:42.941326 4522 tablet_loader.cc:96] loaded metadata for tablet 3c97418290e04f60884c721ded14495a (table default.Table_0752f7e6b5e84c50bb807a28862c4e6b [id=a0f2c50dd24f45528c829ffdc5280653])
I20260810 07:53:42.942175 4522 tablet_loader.cc:96] loaded metadata for tablet 3dfa1b6f604c452e9f346ecc18b5b691 (table default.Table_18cea3a247ec441fa708a7895fb841bc [id=71462c5785c842e1aacc069c1187786d])
I20260810 07:53:42.944427 4522 tablet_loader.cc:96] loaded metadata for tablet 433404aa29574d819371fe090e670095 (table default.Table_b3e30bad905c4837820169afe8a01cd4 [id=d7e7f37d2b7a449cb308fe7b920cd3a9])
I20260810 07:53:42.945447 4522 tablet_loader.cc:96] loaded metadata for tablet 446ed19d914548ea864a43b2bb8c7f5a (table default.Table_f1136e7481f74aaba7d9c2e77b1876f1 [id=73a0c43c4b694dc2b3e2ada66725a4fb])
I20260810 07:53:42.946784 4522 tablet_loader.cc:96] loaded metadata for tablet 44ff256b314e479d981333d9cbfc1656 (table default.Table_55b9e2dd7dc54da1a7a46c5b39f30b69 [id=dbb1dd3d59fb4b2e9b41fd260e416a7b])
I20260810 07:53:42.947628 4522 tablet_loader.cc:96] loaded metadata for tablet 45a3fe9686e0445187c77cf41c563c01 (table default.Table_b7bfd0d718e746c9b7fc82dba4a4c555 [id=5b122a2509cb4c538b1f9e597a6037eb])
I20260810 07:53:42.949471 4522 tablet_loader.cc:96] loaded metadata for tablet 461d70a71f2c44228f4220e7f5fb9c64 (table default.Table_f930ee3bb1ab4902aee3db602c1440ba [id=f3de5e5548744d66bfa60af4cfa7b279])
I20260810 07:53:42.950817 4522 tablet_loader.cc:96] loaded metadata for tablet 4bb34e5b988b497c8f72a6c7266c0f2d (table default.Table_b0c0af5d463242c4acede6ca01119e32 [id=0da5962932774e728b91424ca395431c])
I20260810 07:53:42.952533 4522 tablet_loader.cc:96] loaded metadata for tablet 4d6df1791e5146a38f3cf0304b5034cb (table default.Table_6ff328bbdb89429a8b773fbdc529c27c [id=e814b2a3f97340049d3f30a859fdf079])
I20260810 07:53:42.953965 4522 tablet_loader.cc:96] loaded metadata for tablet 4ecc6870602043269daa61cb173108b3 (table default.Table_09d2a4538e6c41de842dbd63da2067f9 [id=be37d8fb80324008af4429aeec40112b])
I20260810 07:53:42.954967 4522 tablet_loader.cc:96] loaded metadata for tablet 4fb366297938415382cf6555ed052956 (table default.Table_0b54380ef2094266b129d78070f1fcab [id=ff3585263d8a482f994440e089074151])
I20260810 07:53:42.955842 4522 tablet_loader.cc:96] loaded metadata for tablet 505e2f7a25a447c3bf19b310aca1702c (table default.Table_e822693d75fc4bfdad86a06efc56e19c [id=def3910e84cc45e3a788d9d9427f2c8e])
I20260810 07:53:42.958395 4522 tablet_loader.cc:96] loaded metadata for tablet 55aaf296555b43f9a27f2926054c8d4b (table default.Table_f26b16b9c60448cd8f984bf3f106a0f2 [id=2b73ffa465264e3ea5d998b973371acc])
I20260810 07:53:42.959280 4522 tablet_loader.cc:96] loaded metadata for tablet 5a0f23d5212a46cc8f731d9c08019858 (table default.Table_776ba5427fa541f4b2885108ee933c25 [id=3e9b86f943934bd787e7a02164e1efb1])
I20260810 07:53:42.960160 4522 tablet_loader.cc:96] loaded metadata for tablet 5a892c4bc5694118b514cc1baa56601d (table default.Table_30f89a33906a42b9bd617cd1a8d2eaae [id=133ffe64b3e4409ea9e78c9bbe8daea6])
I20260810 07:53:42.962487 4522 tablet_loader.cc:96] loaded metadata for tablet 5d7024a39aa040c2b00196f668b776e8 (table default.Table_371cedf390b0495f84d3da22c3b205ed [id=77bc8e0b324048febec7c6ed6f660ff4])
I20260810 07:53:42.963405 4522 tablet_loader.cc:96] loaded metadata for tablet 5dad896bfa0e4c5cbc4907a75082b064 (table default.Table_2aaec545288b4889811f4c7f91fa7218 [id=73e3632788b8471081f09a42fe124f0e])
I20260810 07:53:42.964814 4522 tablet_loader.cc:96] loaded metadata for tablet 5eb896c8922e412c97e2cc4891298830 (table default.Table_34a402e90a4048849f6098cd5893fe31 [id=06838cb6cc4d41a0be9978b798a2fa19])
W20260810 07:53:42.965655 32565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.139.126:38193 (error 108) [suppressed 1 similar messages]
I20260810 07:53:42.966010 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 32412
I20260810 07:53:42.971779 4522 tablet_loader.cc:96] loaded metadata for tablet 613d1c158c7c4f2f8626536292d00fd1 (table default.Table_0ed10cb37e024390a3b1b7960835aff7 [id=66f6b1a345c14c229a77e35b8a16d133])
I20260810 07:53:42.974013 4522 tablet_loader.cc:96] loaded metadata for tablet 63812f4779914930b4e22d020a51a33b (table default.Table_30fefb1a7d9d4121ba5ad66228a53849 [id=6170cadf04e84e08a82725eb65f7edd9])
I20260810 07:53:42.975310 4522 tablet_loader.cc:96] loaded metadata for tablet 666f36dafa92421cae47c94593ed8899 (table default.Table_088fa7381e654b13b33b9eac210aec71 [id=3e8d6614b9a94df9a1ceb2f7d09c4f3f])
I20260810 07:53:42.977558 4522 tablet_loader.cc:96] loaded metadata for tablet 69f1405e9526433a8fd727ed237f8c84 (table default.Table_d2f777ade76044e5a3f80922d25b9c86 [id=6e2c1c5d301d41acbd95a6e35dbabb23])
I20260810 07:53:42.980752 4522 tablet_loader.cc:96] loaded metadata for tablet 6cd9484084294acfba9613fa49f369b2 (table default.Table_ec9f1bb881a9463cbb641da255034aef [id=a57dab09f7c5419baa978678a309e643])
I20260810 07:53:42.982273 4522 tablet_loader.cc:96] loaded metadata for tablet 6d67fef331994ca48f1dfce1e3b039f4 (table default.Table_d2feab5fe7f043538f85acd989659d91 [id=2fa175ca0ebb4405ad95cff23dca6aa9])
I20260810 07:53:42.983657 4522 tablet_loader.cc:96] loaded metadata for tablet 6db16797cabc48fe8ec82355ce3f16be (table default.Table_7438e01a7d4a499c856ce0f80c8666c9 [id=04ba6086cd9f42ffbc4156b7bf00e250])
W20260810 07:53:42.984390 32565 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2c43f834fbcb447587880136f69c7993 -> Peer 72fde54f23f34224ac756f11924c9531 (127.31.139.126:38193): Couldn't send request to peer 72fde54f23f34224ac756f11924c9531. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38193: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:53:42.985807 4522 tablet_loader.cc:96] loaded metadata for tablet 6f17bb6c7ad74c178e1b5d96b7e688b7 (table default.Table_6b73ad51b4db4578be0a78f07ed60644 [id=151e78c7c4f24a58a5c42aee38110f6e])
I20260810 07:53:42.987509 4522 tablet_loader.cc:96] loaded metadata for tablet 741404076635419bbbd6ce075fc44ccb (table default.Table_87f5c1dc7317431c9068a366c6429c52 [id=843a9b58e035470abfb97dbd62070264])
I20260810 07:53:42.988332 4522 tablet_loader.cc:96] loaded metadata for tablet 74d80c441b2b418994b46250a5c7af44 (table default.Table_5e91c1eeb8ee43adb217b5c6c1dded8c [id=e37ce87d5d5348738d1247fcdc27db24])
I20260810 07:53:42.990080 4522 tablet_loader.cc:96] loaded metadata for tablet 7a1eb27f536648c888a5acc3e940e4c0 (table default.Table_d8243e06c4324d7ba096253015b2eebd [id=3eee7f69234b4dde8cd856bd44a9db70])
I20260810 07:53:42.990967 4522 tablet_loader.cc:96] loaded metadata for tablet 7c075fdf1055477db9f2d56324f12a8a (table default.Table_392a81b62c08432fa1f4a7747cf566f0 [id=d76d144079354771b5455effa4abf6f7])
I20260810 07:53:42.991746 4522 tablet_loader.cc:96] loaded metadata for tablet 7d6c33f86da645e9bf50e45bc229d0a4 (table default.Table_944abc71980d460d94439635f3a18725 [id=13a2b62a847e491ebd5208c432704017])
I20260810 07:53:42.993091 4522 tablet_loader.cc:96] loaded metadata for tablet 82edc39a864f49f8be60e4b36ce3d59c (table default.Table_857eaa5798b14f07bad571642674f2e8 [id=9915d36da31542d0b018cd2463bda067])
I20260810 07:53:42.993876 4522 tablet_loader.cc:96] loaded metadata for tablet 834a48ebb87a4b999722e05af1feab21 (table default.Table_a8e074b8f7eb43ca90782d7de174dc46 [id=003ea48cd1e34ca3a5112ba11394ed83])
I20260810 07:53:42.994669 4522 tablet_loader.cc:96] loaded metadata for tablet 83b68614cd8444a08c7820a33c59442d (table default.Table_f79cf4b4853c4f2d8147136e256dc1aa [id=a90df31804794113ba86e35fceb99a7a])
I20260810 07:53:42.995514 4522 tablet_loader.cc:96] loaded metadata for tablet 849197942f134542a387a10167fc7481 (table default.Table_cedb584fa8d744b5ae31de6b73b34c3b [id=7f033ac75e0647859ae2d4a08a52b1a4])
I20260810 07:53:42.996891 4522 tablet_loader.cc:96] loaded metadata for tablet 8718f998623e4f1c8b7b8be815e184a3 (table default.Table_1a8cb51949d74f3491dbc79f5d488eb6 [id=1083d105acd24b249933790d21f43ef5])
I20260810 07:53:42.998131 4522 tablet_loader.cc:96] loaded metadata for tablet 87ae23cb445e43fc9dc2f45097190fec (table default.Table_28f4942b995147189f6334d58a30f047 [id=1ed5cef38b0740bfabf428d1633eb7de])
I20260810 07:53:42.998960 4522 tablet_loader.cc:96] loaded metadata for tablet 8a95df8c34ad4ff4bfe70cea1e0ebb92 (table default.Table_5f7e619b24fe4837930915dbe61bd2b8 [id=ee979df2f5bc4651a69dfee31d80590f])
I20260810 07:53:42.999747 4522 tablet_loader.cc:96] loaded metadata for tablet 8c02aece0af740828f4009706445e60c (table default.Table_890c4de822874851bd58fd49b7b992ed [id=3aff68ac02f54c30845745acfbb02855])
I20260810 07:53:43.001495 4522 tablet_loader.cc:96] loaded metadata for tablet 8ea09585009e4a3ea33f620182bfee60 (table default.Table_c26400e44124423eb9b7db04cf875aba [id=313cd9f30ae54db38c56df615883fc27])
I20260810 07:53:43.002298 4522 tablet_loader.cc:96] loaded metadata for tablet 8ee78955a68a451f81e01a62a57aa4a4 (table default.Table_94e83c4488294b578216286d2a51ac73 [id=9c057df69fa4417cbc7c256cb7609af7])
I20260810 07:53:43.003080 4522 tablet_loader.cc:96] loaded metadata for tablet 91c2e1bcd13a4c22b2141949b2d73188 (table default.Table_c6abf897fb4e41ad88d667a52e915abf [id=1979f5449bac45598f90219a25c1a44f])
I20260810 07:53:43.003851 4522 tablet_loader.cc:96] loaded metadata for tablet 92c95cd95c034d9a9205221d07ae8615 (table default.Table_a6f9405788014781b2dc4ed324067e30 [id=6c73f64304d74c4a898426e206d7f3c0])
I20260810 07:53:43.005353 4522 tablet_loader.cc:96] loaded metadata for tablet 9350b151227d45e6bae8adddd8fff86b (table default.Table_9d5653ef6b2141efa61961b2fdbed04c [id=f5bade97d7ff4bde89480116a1ca1957])
I20260810 07:53:43.006152 4522 tablet_loader.cc:96] loaded metadata for tablet 9369baf30ab64646b7bc042c6da1987b (table default.Table_227a31b168824c84a470faf1069ad0fa [id=6ed43022a4d74f538afc9f9786e629e1])
I20260810 07:53:43.006932 4522 tablet_loader.cc:96] loaded metadata for tablet 93f62494bb034cd0b4f34f61652569c6 (table default.Table_bb6691e602074986a5e5ecee6fac625e [id=ec1163671c5b429e8c6e9c3354661f7e])
I20260810 07:53:43.007681 4522 tablet_loader.cc:96] loaded metadata for tablet 94ced893d6804aa395b73f1f5ed085f9 (table default.Table_b1983fd910c94cb183eb1c99df54a1a2 [id=fc3b6f6dc27444048db179c49e38cfca])
I20260810 07:53:43.009047 4522 tablet_loader.cc:96] loaded metadata for tablet 98dec9b072ac49959181003cb78567af (table default.Table_19a08db6b4c641dc84fd05c3020827f4 [id=004aa5e06f37451897d26a2e333dfa72])
I20260810 07:53:43.009871 4522 tablet_loader.cc:96] loaded metadata for tablet 999f3e79ee4b4c368b60b202735c2556 (table default.Table_524b25e2037e4ccaa99cbeeca49631f8 [id=0fb6b3cdea154a509c22d09444030fe6])
I20260810 07:53:43.011583 4522 tablet_loader.cc:96] loaded metadata for tablet 9ae857bd2c124682bbd682d81dc591a5 (table default.Table_6cb79839bb98431b93b85916aed4a89b [id=2d4e6609145b4cb78dac8f4cae804c55])
I20260810 07:53:43.012410 4522 tablet_loader.cc:96] loaded metadata for tablet 9b4339a26ad540f4b385f69fd0dcdc5d (table default.Table_ec23fc68529e443b9b2141c44a5140e9 [id=81a9ff900b9141c08d9d0d16e795a661])
I20260810 07:53:43.013319 4522 tablet_loader.cc:96] loaded metadata for tablet 9f6e61aa223048f8bdc9eee07fa2066c (table default.Table_e7b0a1f1654e43f69eac56f97c009155 [id=4ea379ec8bfb40ec83e326b3f1083812])
I20260810 07:53:43.014950 4522 tablet_loader.cc:96] loaded metadata for tablet a3642f29b38241be9db3cf37ccde2039 (table default.Table_c865327885b64e23882fd2d6a58afd0d [id=7ec62f37e4084c2088d8ad764eb10fa1])
I20260810 07:53:43.016167 4522 tablet_loader.cc:96] loaded metadata for tablet a4d1c8d270f744ee9849475c9128414b (table default.Table_efee9f7cc3774e24baf8069ac53455ff [id=69cbb9743df947d1b2d0bc5dd2014614])
I20260810 07:53:43.017455 4522 tablet_loader.cc:96] loaded metadata for tablet a57324aac10d42d191f72e2c2343d4ea (table default.Table_a54449310a7f4844938ec08cfbf054d1 [id=92a2980d1ca145b898bca2bf8f46a99a])
I20260810 07:53:43.018254 4522 tablet_loader.cc:96] loaded metadata for tablet a6077111ffb2411892b62d806ea3c85b (table default.Table_3fdcfe2b3cd74bc9b8350cc7a7cca08e [id=11f4ae5254954640acb2e30536c31426])
I20260810 07:53:43.019055 4522 tablet_loader.cc:96] loaded metadata for tablet a61dbafb4f7444e690cea0d003d38f22 (table default.Table_baa6274b25a64f47bf7c8f7dbc29c6e0 [id=8ebf95685c034249a05157e5d113892d])
I20260810 07:53:43.019809 4522 tablet_loader.cc:96] loaded metadata for tablet a7e077b6b69044c2b6d3d2bb7d99ee29 (table default.Table_ed081fcbe2354d06836e6fd3ee83a434 [id=a0e57381e0534530955acb471b39d793])
I20260810 07:53:43.020582 4522 tablet_loader.cc:96] loaded metadata for tablet a86401e669fa4358ae33ce2249b3a98e (table default.Table_93e4811656944714b25bb9e0de7256c8 [id=6e3a195d72e441d0a3acca4e22f53b45])
I20260810 07:53:43.022238 4522 tablet_loader.cc:96] loaded metadata for tablet a9f5a4321a504ed59e0e2ba21ac801f7 (table default.Table_c21adb23b99c477d9f9ea8bf2fc0558f [id=fe8d440df0df4614925a0a9ea2f18168])
I20260810 07:53:43.023373 4522 tablet_loader.cc:96] loaded metadata for tablet ab2e7619a25b437c8ee2c817255ca638 (table default.Table_a4eef7144b52473388a18de022fc7566 [id=812ee44b494d4dc2b09a70ce3710168b])
I20260810 07:53:43.025074 4522 tablet_loader.cc:96] loaded metadata for tablet aed14fa26b9a46a4b3e9a43bac05a795 (table default.Table_4c14a225e70b402d924a5e3de18dacea [id=b9720089da364d2faa5b69a0dfa3153c])
I20260810 07:53:43.026727 4522 tablet_loader.cc:96] loaded metadata for tablet b2cc9ff5b3344dc39e2e560d63f0ce92 (table default.Table_dc514c4924df483b859282812b30bb1e [id=2a3b7d6510b9471dac3a98c2bf1541bf])
I20260810 07:53:43.027509 4522 tablet_loader.cc:96] loaded metadata for tablet b2fb91a223154dde8ecc2bebf2c0fff9 (table default.Table_ec2bd2c8af504b6c94376bc07ac5946b [id=31222b0775e84831b377a8fd755f6408])
I20260810 07:53:43.028331 4522 tablet_loader.cc:96] loaded metadata for tablet b305a232550e4d4babb19093ea1163f0 (table default.Table_775fb3e64a80463b920fa697ea14223d [id=c47c8b4ab189474ba465129bfca88f5d])
I20260810 07:53:43.029153 4522 tablet_loader.cc:96] loaded metadata for tablet b6226a2377864d47b14901df39677127 (table default.Table_7e58a302aba14a68a5c4281837dbb971 [id=57fbc9f2436f42289d66a57eec7232a9])
I20260810 07:53:43.029909 4522 tablet_loader.cc:96] loaded metadata for tablet b6cb01ce3d97454d9d1e7735e51a76c4 (table default.Table_72d628558a2d41caa3fb7a5baed5ba51 [id=0164360ae1304ca094d8e5f4d4a83c97])
I20260810 07:53:43.030675 4522 tablet_loader.cc:96] loaded metadata for tablet b705e1630f124370af386436f325004b (table default.Table_10b86803e3984d76bdb6f83b57a9451f [id=bb1231d8e5524d3eb6e7f994ea6f1257])
I20260810 07:53:43.031445 4522 tablet_loader.cc:96] loaded metadata for tablet b7bbccbecf884768b3b171d9bd901b9b (table default.Table_110c350dc5474f2fbe9bbd9410aac109 [id=b23efee8dd99419d9308ef2d5a907409])
I20260810 07:53:43.032233 4522 tablet_loader.cc:96] loaded metadata for tablet bdc0039db22d4812a1412e10175d2237 (table default.Table_15c4a71f6b934bd58563002b8e876760 [id=e82c5e6520074475b6ba5f6f6ec0a4e5])
I20260810 07:53:43.033074 4522 tablet_loader.cc:96] loaded metadata for tablet bf6663cbe7fd43a79265d87c2cc11a62 (table default.Table_f517aea30ffe40f2922cdae2d9712eb0 [id=4ba744378de040d0a38e2f18220e69c5])
I20260810 07:53:43.033926 4522 tablet_loader.cc:96] loaded metadata for tablet c1826ebc26f04805b70ac5314e48fda9 (table default.Table_6d66b8948fe749769480f2a7c912be17 [id=e11987dccb6140aaa52a5ad9d2f5e2da])
I20260810 07:53:43.036505 4522 tablet_loader.cc:96] loaded metadata for tablet c47792551bb243b7b3f6de904c17c47b (table default.Table_f77039cfd411423da5e7bf5e02a6aa81 [id=5066a36786d04f72b0ba19bc5145322a])
I20260810 07:53:43.037344 4522 tablet_loader.cc:96] loaded metadata for tablet c592efc3acbf49659ecd7d7acb2881e5 (table default.Table_1315259f9b4e4425a2f99c5125e402bc [id=3b6a4e9c10744d4dbef52ed4a87fc36f])
I20260810 07:53:43.038247 4522 tablet_loader.cc:96] loaded metadata for tablet c6080c8b12014fe09d48e307d394f9c9 (table default.Table_f3d034925e2e4dd6b7b0afff1c67e87d [id=497153183b2143b498370554caae46b4])
I20260810 07:53:43.040056 4522 tablet_loader.cc:96] loaded metadata for tablet cc9959255b4d46e29b9164027638e38e (table default.Table_0539c0c6e0054726834e1dfe114e4ef7 [id=5ab60d4288824fc08a28a5e180eca866])
I20260810 07:53:43.040907 4522 tablet_loader.cc:96] loaded metadata for tablet cca378598d0c45178e497fd212638d77 (table default.Table_5a9f797e42664156b0addc3bbf49ffc2 [id=1d81b0ab4d9e4f06b06d15cc1aaf7995])
I20260810 07:53:43.042958 4522 tablet_loader.cc:96] loaded metadata for tablet d3077b3fe04e406a89560229b76d8242 (table default.Table_ec03572c5d844816b4b09051f198f554 [id=69cc244072194e24af38b77cc5ccce0f])
I20260810 07:53:43.045816 4522 tablet_loader.cc:96] loaded metadata for tablet dd79131856f54395a3c5fb551961d4f7 (table default.Table_8d4b32c5738e41c197473d44c3151c37 [id=72d66e05fa74407f9576a1d5bd59ee3d])
I20260810 07:53:43.046650 4522 tablet_loader.cc:96] loaded metadata for tablet e0a74b22a6864180956e763d2b6b6de6 (table default.Table_c0dcb3ae0d0a43d2834f4db818d5a54e [id=a113e7150c8849e0a86355b8fc4a1b92])
I20260810 07:53:43.048290 4522 tablet_loader.cc:96] loaded metadata for tablet e20d46d62cdc48669d7474fae00660d7 (table default.Table_aff6b5cbe0e8421da86a454f107e16d2 [id=eb01e542c4d44028a311127f924d1446])
I20260810 07:53:43.049121 4522 tablet_loader.cc:96] loaded metadata for tablet e2fe3153512b40a99caf3fb9b81fd7a1 (table default.Table_77e86fb7c5274253ab027b7dec950ae0 [id=f1ce7fa2a6c542e3a1882f2cde3d4fb8])
I20260810 07:53:43.050244 4522 tablet_loader.cc:96] loaded metadata for tablet e38ab80cd5c441f3ba20bf00577b223c (table default.Table_5152115799f74951b88c4268d7aeeec9 [id=5670824bda1343beaecc40639441fc18])
I20260810 07:53:43.052314 4522 tablet_loader.cc:96] loaded metadata for tablet e7529f4ac9fb457ba1b3c3bc90eb275d (table default.Table_6261a75ba7d54f049285cbd6d8d5b878 [id=6be5cfed723b4a63ba1e0190d3afd898])
I20260810 07:53:43.054399 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 32526
I20260810 07:53:43.054404 4522 tablet_loader.cc:96] loaded metadata for tablet ebfc73a83a0f4048ad9dded62d074e19 (table default.Table_8e034abdc8d141e4bc8c80614ba099f9 [id=2e99f293b37a4c6fa49f4c8dee1c7f43])
2026-08-10T07:53:43Z chronyd exiting
I20260810 07:53:43.121279 32301 master-stress-test.cc:587] Tables created: 175
I20260810 07:53:43.121541 32301 master-stress-test.cc:588] Tables altered: 69
I20260810 07:53:43.121791 32301 master-stress-test.cc:589] Tables deleted: 55
I20260810 07:53:43.121982 32301 master-stress-test.cc:590] Tablets replaced: 37
I20260810 07:53:43.122167 32301 master-stress-test.cc:591] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (73338 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-08-10T07:53:43Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:53:43Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdk4jGL/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdk4jGL/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/security.properties
2026-08-10 07:53:51: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdk4jGL/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdk4jGL/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260810 07:54:01.370142 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38803
--webserver_interface=127.31.139.126
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--hive_metastore_uris=thrift://127.0.0.1:43293
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 07:54:01.692853 4979 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:01.693536 4979 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:01.694219 4979 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:01.728418 4979 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260810 07:54:01.728724 4979 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260810 07:54:01.729061 4979 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:54:01.729339 4979 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:01.729605 4979 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:54:01.729876 4979 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:54:01.769006 4979 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:43293
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.139.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:01.770512 4979 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:01.772197 4979 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:54:01.784049 4989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:01.784251 4990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:03.031261 4992 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1241 milliseconds
W20260810 07:54:03.034221 4993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:03.036279 4979 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.252s user 0.003s sys 0.006s
W20260810 07:54:03.036679 4979 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.253s user 0.003s sys 0.006s
I20260810 07:54:03.037034 4979 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:03.038758 4979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:03.042253 4979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:03.043838 4979 hybrid_clock.cc:648] HybridClock initialized: now 1786348443043781 us; error 48 us; skew 500 ppm
I20260810 07:54:03.045167 4979 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:03.055074 4979 webserver.cc:533] Webserver started at http://127.31.139.126:37953/ using document root <none> and password file <none>
I20260810 07:54:03.056697 4979 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:03.057083 4979 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:03.057787 4979 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:03.065229 4979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/instance:
uuid: "33049f5d320f4c6ca8d040c13fa09252"
format_stamp: "Formatted at 2026-08-10 07:54:03 on dist-test-slave-x5fp"
I20260810 07:54:03.066982 4979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal/instance:
uuid: "33049f5d320f4c6ca8d040c13fa09252"
format_stamp: "Formatted at 2026-08-10 07:54:03 on dist-test-slave-x5fp"
I20260810 07:54:03.078537 4979 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.011s sys 0.001s
I20260810 07:54:03.087102 5005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:03.088667 4979 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 07:54:03.089111 4979 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
uuid: "33049f5d320f4c6ca8d040c13fa09252"
format_stamp: "Formatted at 2026-08-10 07:54:03 on dist-test-slave-x5fp"
I20260810 07:54:03.089497 4979 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:03.160236 4979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:03.161859 4979 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:03.162354 4979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:03.239436 4979 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.126:38803
I20260810 07:54:03.239528 5087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.126:38803 every 8 connection(s)
I20260810 07:54:03.242296 4979 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
I20260810 07:54:03.248488 5088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:03.252416 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 4979
I20260810 07:54:03.253000 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal/instance
I20260810 07:54:03.256620 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.125:41987
--webserver_interface=127.31.139.125
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--hive_metastore_uris=thrift://127.0.0.1:43293
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:54:03.265441 5088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:03.298238 5088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:54:03.301154 5008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:03.305699 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:03.355294 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 1
W20260810 07:54:03.360096 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:03.428635 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 2
W20260810 07:54:03.433139 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:03.524580 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 3
W20260810 07:54:03.529424 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:03.595873 5090 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:03.596537 5090 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:03.597021 5090 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:03.630620 5090 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260810 07:54:03.630916 5090 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260810 07:54:03.631228 5090 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:54:03.631502 5090 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:03.631771 5090 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:54:03.632041 5090 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:54:03.669286 5090 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:43293
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.125:41987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.139.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:03.670737 5090 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:03.672482 5090 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260810 07:54:03.672907 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 4
W20260810 07:54:03.677197 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:03.686528 5100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:03.976624 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 5
W20260810 07:54:03.984194 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:04.531888 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 6
W20260810 07:54:04.545207 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:03.687108 5101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:04.935510 5107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:04.939764 5103 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1251 milliseconds
I20260810 07:54:04.939860 5090 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:04.941457 5090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:04.944335 5090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:04.945750 5090 hybrid_clock.cc:648] HybridClock initialized: now 1786348444945711 us; error 41 us; skew 500 ppm
I20260810 07:54:04.946650 5090 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:04.953248 5090 webserver.cc:533] Webserver started at http://127.31.139.125:46401/ using document root <none> and password file <none>
I20260810 07:54:04.954259 5090 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:04.954519 5090 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:04.955103 5090 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:04.959746 5090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data/instance:
uuid: "c4cb37d55c894071b60fe8c11447e918"
format_stamp: "Formatted at 2026-08-10 07:54:04 on dist-test-slave-x5fp"
I20260810 07:54:04.961038 5090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal/instance:
uuid: "c4cb37d55c894071b60fe8c11447e918"
format_stamp: "Formatted at 2026-08-10 07:54:04 on dist-test-slave-x5fp"
I20260810 07:54:04.969100 5090 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 07:54:04.975342 5121 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:04.976794 5090 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.004s
I20260810 07:54:04.977205 5090 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal
uuid: "c4cb37d55c894071b60fe8c11447e918"
format_stamp: "Formatted at 2026-08-10 07:54:04 on dist-test-slave-x5fp"
I20260810 07:54:04.977542 5090 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:05.029686 5090 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:05.031426 5090 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:05.032022 5090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:05.110329 5090 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.125:41987
I20260810 07:54:05.110414 5192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.125:41987 every 8 connection(s)
I20260810 07:54:05.113227 5090 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/data/info.pb
I20260810 07:54:05.117133 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 5090
I20260810 07:54:05.117594 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-1/wal/instance
I20260810 07:54:05.119769 5193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:05.120168 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:41801
--webserver_interface=127.31.139.124
--webserver_port=0
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--hive_metastore_uris=thrift://127.0.0.1:43293
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:54:05.130599 5193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:05.159032 5193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:05.174652 5193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:54:05.177484 5125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:05.182288 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.231997 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 1
W20260810 07:54:05.238225 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.306741 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 2
W20260810 07:54:05.312811 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.404345 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 3
W20260810 07:54:05.409245 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:05.511782 5194 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:05.512439 5194 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:05.513010 5194 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:05.547324 5194 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260810 07:54:05.547675 5194 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260810 07:54:05.547957 5194 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:54:05.548220 5194 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:05.548485 5194 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:54:05.548810 5194 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:54:05.552680 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 4
W20260810 07:54:05.557296 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.589191 5194 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:43293
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:41801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.139.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:05.590657 5194 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:05.592428 5194 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:54:05.603649 5211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:05.605821 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } attempt: 7
I20260810 07:54:05.627784 5088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
W20260810 07:54:05.649150 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.707796 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 1
W20260810 07:54:05.716584 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.798345 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 2
W20260810 07:54:05.806363 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.856808 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 5
W20260810 07:54:05.865481 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:05.891968 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 3
W20260810 07:54:05.901813 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:06.042521 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 4
W20260810 07:54:06.051021 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:06.334619 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 5
W20260810 07:54:06.343150 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:06.413076 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 6
W20260810 07:54:06.440809 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:06.895763 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 6
W20260810 07:54:06.923076 5088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:05.605420 5212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:07.008610 5210 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee60 after lost signal to thread 5194
W20260810 07:54:07.330587 5194 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.726s user 0.565s sys 1.146s
W20260810 07:54:07.330963 5194 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.727s user 0.565s sys 1.146s
W20260810 07:54:07.335393 5214 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1727 milliseconds
W20260810 07:54:07.336899 5215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:07.336997 5194 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:07.338407 5194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:07.341158 5194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:07.342584 5194 hybrid_clock.cc:648] HybridClock initialized: now 1786348447342542 us; error 39 us; skew 500 ppm
I20260810 07:54:07.343559 5194 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:07.350658 5194 webserver.cc:533] Webserver started at http://127.31.139.124:44291/ using document root <none> and password file <none>
I20260810 07:54:07.351780 5194 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:07.352053 5194 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:07.352581 5194 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:07.357336 5194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/instance:
uuid: "a552a496f9e0418d97a2006bdef0af5a"
format_stamp: "Formatted at 2026-08-10 07:54:07 on dist-test-slave-x5fp"
I20260810 07:54:07.358578 5194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal/instance:
uuid: "a552a496f9e0418d97a2006bdef0af5a"
format_stamp: "Formatted at 2026-08-10 07:54:07 on dist-test-slave-x5fp"
I20260810 07:54:07.366850 5194 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260810 07:54:07.373409 5243 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:07.374567 5194 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:54:07.374969 5194 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
uuid: "a552a496f9e0418d97a2006bdef0af5a"
format_stamp: "Formatted at 2026-08-10 07:54:07 on dist-test-slave-x5fp"
I20260810 07:54:07.375349 5194 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:07.436409 5194 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:07.438181 5194 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:07.438680 5194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:07.501497 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 7
W20260810 07:54:07.510293 5193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.31.139.124:41801: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:07.518389 5194 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.124:41801
I20260810 07:54:07.518482 5331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.124:41801 every 8 connection(s)
I20260810 07:54:07.521400 5194 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
I20260810 07:54:07.527585 5332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:07.530261 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 5194
I20260810 07:54:07.530723 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal/instance
I20260810 07:54:07.544318 5332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:07.570216 5332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:07.581555 5332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:54:07.600076 5332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Bootstrap starting.
I20260810 07:54:07.606274 5332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:07.608666 5332 log.cc:826] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:07.614336 5332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: No bootstrap required, opened a new log
I20260810 07:54:07.633395 5332 raft_consensus.cc:359] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:07.634030 5332 raft_consensus.cc:385] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:07.634239 5332 raft_consensus.cc:740] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a552a496f9e0418d97a2006bdef0af5a, State: Initialized, Role: FOLLOWER
I20260810 07:54:07.635039 5332 consensus_queue.cc:260] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:07.637535 5342 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:07.638180 5342 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:07.638960 5332 sys_catalog.cc:565] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:54:07.677394 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.679903 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.694074 5251 tablet.cc:2406] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:07.666299 5194 master_runner.cc:187] Error getting master registration for 127.31.139.126:38803: OK, instance_id { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" instance_seqno: 1786348443217441 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260810 07:54:07.709033 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 07:54:07.705098 5194 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:54:07.712875 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.727734 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.728134 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.742835 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.743180 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.757616 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.757963 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.772845 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.773177 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.787611 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.787949 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.802450 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.802802 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.817520 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.817927 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.832523 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.832906 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.848359 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.848703 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.863168 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.863504 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.877921 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.878281 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.892889 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.893245 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.907764 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.908125 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.924273 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.924701 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 07:54:07.939165 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.939513 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:07.951522 5342 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:07.952244 5342 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
W20260810 07:54:07.955185 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.955579 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:07.956956 5342 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
I20260810 07:54:07.958452 5088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 7
W20260810 07:54:07.974819 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.975255 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:07.988915 5088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Bootstrap starting.
W20260810 07:54:07.993158 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:07.993814 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.002460 5088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:08.005425 5088 log.cc:826] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Log is configured to *not* fsync() on all Append() calls
W20260810 07:54:08.010202 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:08.010937 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.013314 5088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: No bootstrap required, opened a new log
W20260810 07:54:08.034818 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:08.035992 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.038079 5088 raft_consensus.cc:359] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:08.038820 5088 raft_consensus.cc:385] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:08.039067 5088 raft_consensus.cc:740] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33049f5d320f4c6ca8d040c13fa09252, State: Initialized, Role: FOLLOWER
I20260810 07:54:08.040131 5088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:08.043627 5371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:08.044657 5371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:08.045910 5088 sys_catalog.cc:565] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:54:08.046149 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:08.047051 5050 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a552a496f9e0418d97a2006bdef0af5a in term 0.
I20260810 07:54:08.048686 5247 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a; no voters:
I20260810 07:54:08.049922 5342 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:08.050266 5342 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:08.050594 5342 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:08.057662 5342 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:08.062183 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252"
I20260810 07:54:08.063539 5050 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:08.070925 5050 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a552a496f9e0418d97a2006bdef0af5a in term 1.
I20260810 07:54:08.076826 5342 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 election: Requested vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
W20260810 07:54:08.078694 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 07:54:08.078393 5247 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a; no voters:
W20260810 07:54:08.079089 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.079622 5342 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:08.082628 5342 raft_consensus.cc:697] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 LEADER]: Becoming Leader. State: Replica: a552a496f9e0418d97a2006bdef0af5a, State: Running, Role: LEADER
I20260810 07:54:08.083710 5342 consensus_queue.cc:237] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:08.092540 5376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552a496f9e0418d97a2006bdef0af5a. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:08.098834 5376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: LEADER
W20260810 07:54:08.098415 5364 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:54:08.100466 5364 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.107275 5383 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:54:08.116369 5383 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:54:08.137988 5050 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Refusing update from remote peer a552a496f9e0418d97a2006bdef0af5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:54:08.140026 5376 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:54:08.174003 5014 tablet.cc:2406] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:08.179369 5371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552a496f9e0418d97a2006bdef0af5a. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:08.180389 5371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:08.200203 5376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:08.200979 5376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:08.202075 5383 catalog_manager.cc:1383] Generated new cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:08.202394 5383 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:54:08.220033 5408 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:54:08.226709 5371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:08.227453 5371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:08.270027 5408 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:08.278937 5383 catalog_manager.cc:1406] Generated new certificate authority record
I20260810 07:54:08.281358 5383 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:54:08.281395 5408 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: loading cluster ID for follower catalog manager: success
W20260810 07:54:08.286191 5408 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 07:54:08.286654 5408 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.230095 4979 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: OK, instance_id { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" instance_seqno: 1786348445089167 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:54:08.289386 4979 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:08.305831 5408 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: acquiring CA information for follower catalog manager: success
W20260810 07:54:08.314185 5408 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260810 07:54:08.324474 5383 catalog_manager.cc:6092] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Generated new TSK 0
I20260810 07:54:08.325749 5383 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:54:08.335067 5383 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 07:54:08.339355 5383 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 07:54:08.356441 5408 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:54:08.380110 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.65:0
--local_ip_for_outbound_sockets=127.31.139.65
--webserver_interface=127.31.139.65
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260810 07:54:08.537937 5246 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.139.125:41987 timed out after 0.580s (SENT)
W20260810 07:54:09.128460 5246 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Timed out: RequestConsensusVote RPC to 127.31.139.125:41987 timed out after 0.580s (SENT)
W20260810 07:54:09.254082 5410 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:09.254794 5410 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:09.255726 5410 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:09.361034 5410 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:09.362594 5410 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.65
W20260810 07:54:09.406149 5246 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Timed out: RequestConsensusVote RPC to 127.31.139.125:41987 timed out after 0.664s (SENT)
I20260810 07:54:09.437134 5410 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.139.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.65
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:09.438932 5410 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:09.441203 5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:54:09.468592 5418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:09.600708 5193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } attempt: 8
I20260810 07:54:09.867259 5193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Bootstrap starting.
I20260810 07:54:09.889189 5193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:09.891922 5193 log.cc:826] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:09.911193 5193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: No bootstrap required, opened a new log
I20260810 07:54:09.973325 5193 raft_consensus.cc:359] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:09.974404 5193 raft_consensus.cc:385] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:09.974809 5193 raft_consensus.cc:740] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cb37d55c894071b60fe8c11447e918, State: Initialized, Role: FOLLOWER
I20260810 07:54:09.976033 5193 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:10.028234 5193 sys_catalog.cc:565] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:54:10.028494 5157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918"
I20260810 07:54:10.029942 5154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918"
W20260810 07:54:10.031121 5154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38238 (request call id 5) took 1289 ms (client timeout 664 ms). Trace:
W20260810 07:54:10.031545 5154 rpcz_store.cc:269] 0810 07:54:08.741836 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:08.741989 (+ 153us) service_pool.cc:224] Handling call
0810 07:54:10.031021 (+1289032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:54:10.034272 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
W20260810 07:54:10.035348 5158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38238 (request call id 1) took 2077 ms (client timeout 580 ms). Trace:
W20260810 07:54:10.035897 5158 rpcz_store.cc:269] 0810 07:54:07.958258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:07.958407 (+ 149us) service_pool.cc:224] Handling call
0810 07:54:10.035251 (+2076844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:54:10.036958 5155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
W20260810 07:54:10.039354 5155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38238 (request call id 4) took 1491 ms (client timeout 580 ms). Trace:
W20260810 07:54:10.039768 5155 rpcz_store.cc:269] 0810 07:54:08.547943 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:08.548091 (+ 148us) service_pool.cc:224] Handling call
0810 07:54:10.039263 (+1491172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260810 07:54:10.043043 5157 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:54:10.043690 5129 tablet.cc:2406] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:10.054054 5157 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a552a496f9e0418d97a2006bdef0af5a in term 1.
I20260810 07:54:10.060438 5430 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:10.069437 5430 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:10.077140 5156 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Refusing update from remote peer a552a496f9e0418d97a2006bdef0af5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:54:10.079327 5156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38238 (request call id 3) took 1942 ms. Trace:
I20260810 07:54:10.079696 5156 rpcz_store.cc:276] 0810 07:54:08.136495 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:08.136630 (+ 135us) service_pool.cc:224] Handling call
0810 07:54:10.076483 (+1939853us) raft_consensus.cc:1471] Updating replica for 0 ops
0810 07:54:10.077550 (+ 1067us) raft_consensus.cc:1697] Filling consensus response to leader.
0810 07:54:10.078982 (+ 1432us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5400030550. stack: 00007f59ba9d9cf3 00007f59ba9d9bf0 00007f59ba587574 0000000000516de4 0000000000516d79 00007f59c1ada951 00007f59c1b70c8d 00007f59c2bcf229 00007f59c124dd34 00007f59c124dcc2 00007f59c124dbf1 00007f59c124db6c 00007f59c124ce22 00007f59bf83c07c 00007f59bf83b4b6 00007f59bf83ae5f
0810 07:54:10.079255 (+ 273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107403136}
I20260810 07:54:10.083580 5342 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:54:10.101630 5157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38238 (request call id 2) took 2026 ms (client timeout 664 ms). Trace:
W20260810 07:54:10.102686 5157 rpcz_store.cc:269] 0810 07:54:08.074732 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:08.074859 (+ 127us) service_pool.cc:224] Handling call
0810 07:54:10.076201 (+2001342us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b5400030554. stack: 00007f59ba9d9cf3 00007f59ba9d9bf0 00007f59ba587574 0000000000516de4 0000000000516d79 00007f59c1ada951 00007f59c1b756da 00007f59c2bd0927 00007f59c1252834 00007f59c12527c2 00007f59c12526f1 00007f59c125266c 00007f59c1251922 00007f59bf83c07c 00007f59bf83b4b6 00007f59bf83ae5f
0810 07:54:10.101516 (+ 25315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31318400}
I20260810 07:54:10.176862 5430 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552a496f9e0418d97a2006bdef0af5a. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:10.177636 5430 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:10.179078 5456 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:10.179961 5456 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:10.220050 5465 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:54:10.225368 5461 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:10.227219 5461 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:10.235888 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:10.245882 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:10.249099 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:10.263257 5461 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:10.265321 5461 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:10.272805 5465 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:10.273267 5465 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: loading cluster ID for follower catalog manager: success
I20260810 07:54:10.296370 5465 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: acquiring CA information for follower catalog manager: success
I20260810 07:54:10.305222 5465 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:54:10.538396 5342 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:10.539158 5342 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:12.016305 5477 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:12.018273 5477 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:12.023608 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:12.023929 5477 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:12.027192 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:12.030247 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:12.031298 5477 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:10.867575 5416 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 5410
W20260810 07:54:12.045359 5410 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 2.571s user 0.644s sys 1.050s
W20260810 07:54:12.049911 5410 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 2.576s user 0.644s sys 1.051s
W20260810 07:54:09.476382 5419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:12.054654 5410 server_base.cc:1061] running on GCE node
W20260810 07:54:12.055622 5482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:12.060165 5410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:12.063030 5410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:12.064667 5410 hybrid_clock.cc:648] HybridClock initialized: now 1786348452064617 us; error 42 us; skew 500 ppm
I20260810 07:54:12.066035 5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:12.075757 5410 webserver.cc:533] Webserver started at http://127.31.139.65:41901/ using document root <none> and password file <none>
I20260810 07:54:12.077138 5410 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:12.077440 5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:12.078016 5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:12.084666 5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data/instance:
uuid: "d90b26857dd7400b8f9dbd9e3216dca3"
format_stamp: "Formatted at 2026-08-10 07:54:12 on dist-test-slave-x5fp"
I20260810 07:54:12.086339 5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/instance:
uuid: "d90b26857dd7400b8f9dbd9e3216dca3"
format_stamp: "Formatted at 2026-08-10 07:54:12 on dist-test-slave-x5fp"
I20260810 07:54:12.097488 5410 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.005s sys 0.005s
I20260810 07:54:12.106050 5488 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:12.107650 5410 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260810 07:54:12.108111 5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal
uuid: "d90b26857dd7400b8f9dbd9e3216dca3"
format_stamp: "Formatted at 2026-08-10 07:54:12 on dist-test-slave-x5fp"
I20260810 07:54:12.108568 5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:12.190001 5410 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:12.191778 5410 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:12.192310 5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:12.195873 5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:54:12.201527 5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:54:12.201750 5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:12.202102 5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:54:12.202257 5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:12.344274 5613 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:12.344975 5613 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:12.348610 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:12.350061 5613 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:12.352341 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:12.353888 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:12.354859 5613 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:12.402609 5410 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.65:44779
I20260810 07:54:12.402729 5665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.65:44779 every 8 connection(s)
I20260810 07:54:12.405699 5410 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/data/info.pb
I20260810 07:54:12.411485 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 5410
I20260810 07:54:12.412289 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/instance
I20260810 07:54:12.431037 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.66:0
--local_ip_for_outbound_sockets=127.31.139.66
--webserver_interface=127.31.139.66
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 07:54:12.482365 5673 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:12.482970 5673 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:12.484580 5673 heartbeater.cc:507] Master 127.31.139.126:38803 requested a full tablet report, sending...
I20260810 07:54:12.488157 5032 ts_manager.cc:194] Registered new tserver with Master: d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:12.492214 5668 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:12.492617 5668 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:12.493595 5668 heartbeater.cc:507] Master 127.31.139.124:41801 requested a full tablet report, sending...
I20260810 07:54:12.496469 5266 ts_manager.cc:194] Registered new tserver with Master: d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:12.498849 5666 heartbeater.cc:344] Connected to a master server at 127.31.139.125:41987
I20260810 07:54:12.499203 5666 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:12.499416 5266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.65:50237
I20260810 07:54:12.500114 5666 heartbeater.cc:507] Master 127.31.139.125:41987 requested a full tablet report, sending...
I20260810 07:54:12.504024 5147 ts_manager.cc:194] Registered new tserver with Master: d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:12.560271 5682 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:12.560864 5682 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:12.563457 5682 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:12.563673 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:12.564853 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:12.566401 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:12.567673 5682 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:12.827186 5680 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:12.827751 5680 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:12.828771 5680 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:12.865406 5680 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:12.866485 5680 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.66
I20260810 07:54:12.906927 5680 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.139.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.66
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:12.908391 5680 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:12.910127 5680 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260810 07:54:12.911789 5688 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:12.912284 5688 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:12.914700 5688 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:12.916085 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:12.916069 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:12.917996 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:12.918785 5688 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:12.928846 5690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:13.006531 5668 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
I20260810 07:54:13.947142 5700 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:13.948096 5700 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:13.955538 5700 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:13.962180 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:14.016855 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:14.020414 5008 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:14.021598 5700 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:14.022774 5701 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:14.027666 5701 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:14.043728 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:14.049614 5701 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:14.049749 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:14.051759 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:14.053474 5701 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:14.331015 5689 debug-util.cc:398] Leaking SignalData structure 0x7b0800037500 after lost signal to thread 5680
W20260810 07:54:14.977316 5680 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 2.047s user 0.707s sys 1.217s
W20260810 07:54:14.979072 5680 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 2.049s user 0.707s sys 1.218s
W20260810 07:54:12.929725 5693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:14.980103 5694 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 2048 milliseconds
W20260810 07:54:14.980527 5696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:14.980418 5680 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:14.984002 5680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:14.986387 5680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:14.987731 5680 hybrid_clock.cc:648] HybridClock initialized: now 1786348454987704 us; error 43 us; skew 500 ppm
I20260810 07:54:14.988643 5680 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:14.994989 5680 webserver.cc:533] Webserver started at http://127.31.139.66:38935/ using document root <none> and password file <none>
I20260810 07:54:14.996001 5680 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:14.996234 5680 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:14.996779 5680 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:15.001653 5680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data/instance:
uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
format_stamp: "Formatted at 2026-08-10 07:54:14 on dist-test-slave-x5fp"
I20260810 07:54:15.002846 5680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/instance:
uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
format_stamp: "Formatted at 2026-08-10 07:54:14 on dist-test-slave-x5fp"
I20260810 07:54:15.010670 5680 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260810 07:54:15.017994 5709 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:15.019613 5680 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260810 07:54:15.020007 5680 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal
uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
format_stamp: "Formatted at 2026-08-10 07:54:14 on dist-test-slave-x5fp"
I20260810 07:54:15.020329 5680 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:15.085470 5680 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:15.087311 5680 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:15.087862 5680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:15.090965 5680 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:54:15.096347 5680 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:54:15.096580 5680 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:15.096920 5680 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:54:15.097108 5680 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:15.274756 5895 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:15.275681 5680 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.66:38495
I20260810 07:54:15.275400 5895 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:15.275846 5901 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.66:38495 every 8 connection(s)
I20260810 07:54:15.277244 5895 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:15.278370 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:15.278656 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:15.279665 5680 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/data/info.pb
I20260810 07:54:15.281288 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:15.282466 5895 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:15.286818 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 5680
I20260810 07:54:15.287318 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/instance
I20260810 07:54:15.320364 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.139.67:0
--local_ip_for_outbound_sockets=127.31.139.67
--webserver_interface=127.31.139.67
--webserver_port=0
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260810 07:54:15.343333 5902 heartbeater.cc:344] Connected to a master server at 127.31.139.125:41987
I20260810 07:54:15.343894 5902 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:15.345568 5902 heartbeater.cc:507] Master 127.31.139.125:41987 requested a full tablet report, sending...
I20260810 07:54:15.348049 5904 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:15.348737 5904 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:15.349871 5904 heartbeater.cc:507] Master 127.31.139.124:41801 requested a full tablet report, sending...
I20260810 07:54:15.352481 5266 ts_manager.cc:194] Registered new tserver with Master: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:15.352490 5147 ts_manager.cc:194] Registered new tserver with Master: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:15.354212 5266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.66:59837
I20260810 07:54:15.359480 5905 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:15.359817 5905 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:15.360357 5905 heartbeater.cc:507] Master 127.31.139.126:38803 requested a full tablet report, sending...
I20260810 07:54:15.362625 5032 ts_manager.cc:194] Registered new tserver with Master: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:15.456667 5915 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:15.457144 5915 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:15.458683 5915 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:15.459757 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:15.459882 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:15.461707 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:15.462577 5915 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:15.718524 5914 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:15.719214 5914 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:15.719759 5914 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:15.754350 5914 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:15.755364 5914 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.139.67
I20260810 07:54:15.795665 5914 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.139.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.139.67
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:15.797211 5914 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:15.798950 5914 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:54:15.812115 5925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:15.861789 5904 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
I20260810 07:54:16.076653 5933 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:16.077484 5933 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:16.081106 5933 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:16.082953 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:16.085587 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:16.089200 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:16.090780 5933 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:16.305570 5935 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:16.306218 5935 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:16.308866 5935 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:16.311100 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:16.311213 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:16.314491 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:16.316373 5935 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:16.616751 5938 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:16.617570 5938 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:16.621770 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:16.622031 5938 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:16.624962 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:16.626997 5009 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:16.628207 5938 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:16.908355 5939 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:16.909143 5939 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:16.911978 5939 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:16.914029 5281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:16.914131 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:16.917248 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:16.918851 5939 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260810 07:54:15.821017 5928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:15.814038 5926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:17.192945 5927 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1374 milliseconds
I20260810 07:54:17.193113 5914 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:17.194484 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:17.197264 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:17.198670 5914 hybrid_clock.cc:648] HybridClock initialized: now 1786348457198621 us; error 60 us; skew 500 ppm
I20260810 07:54:17.199570 5914 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:17.207231 5914 webserver.cc:533] Webserver started at http://127.31.139.67:38669/ using document root <none> and password file <none>
I20260810 07:54:17.208611 5914 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:17.208942 5914 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:17.209856 5914 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:54:17.215364 5914 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data/instance:
uuid: "5c7100eee0ce4dae895ca9535696dfce"
format_stamp: "Formatted at 2026-08-10 07:54:17 on dist-test-slave-x5fp"
I20260810 07:54:17.216646 5914 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/instance:
uuid: "5c7100eee0ce4dae895ca9535696dfce"
format_stamp: "Formatted at 2026-08-10 07:54:17 on dist-test-slave-x5fp"
I20260810 07:54:17.225714 5914 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260810 07:54:17.232955 5948 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:17.234315 5914 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260810 07:54:17.234726 5914 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal
uuid: "5c7100eee0ce4dae895ca9535696dfce"
format_stamp: "Formatted at 2026-08-10 07:54:17 on dist-test-slave-x5fp"
I20260810 07:54:17.235108 5914 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:17.294891 5914 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:17.296777 5914 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:17.297376 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:17.300168 5914 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260810 07:54:17.304581 5914 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:54:17.304821 5914 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 07:54:17.305142 5914 ts_tablet_manager.cc:616] Registered 0 tablets
I20260810 07:54:17.305335 5914 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:17.465047 5914 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.67:37449
I20260810 07:54:17.465142 6129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.67:37449 every 8 connection(s)
I20260810 07:54:17.467880 5914 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/data/info.pb
I20260810 07:54:17.476068 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 5914
I20260810 07:54:17.476940 32301 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/instance
I20260810 07:54:17.531497 6138 heartbeater.cc:344] Connected to a master server at 127.31.139.125:41987
I20260810 07:54:17.532258 6138 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:17.533907 6138 heartbeater.cc:507] Master 127.31.139.125:41987 requested a full tablet report, sending...
I20260810 07:54:17.536203 6139 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:17.536609 6139 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:17.537565 6139 heartbeater.cc:507] Master 127.31.139.124:41801 requested a full tablet report, sending...
I20260810 07:54:17.538093 5147 ts_manager.cc:194] Registered new tserver with Master: 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:17.540053 6140 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:17.540405 6140 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:17.540989 5266 ts_manager.cc:194] Registered new tserver with Master: 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:17.541460 6140 heartbeater.cc:507] Master 127.31.139.126:38803 requested a full tablet report, sending...
I20260810 07:54:17.542896 5266 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.31.139.67:41461
I20260810 07:54:17.545094 5032 ts_manager.cc:194] Registered new tserver with Master: 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:17.557682 32301 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260810 07:54:17.608016 5265 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46582:
name: "default.Table_ea1051c6abd749979cb4e6c577078521"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:17.607514 5266 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46582:
name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:17.609627 5264 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46582:
name: "default.Table_d72b29e6aad6422384022c20cf87f343"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:17.614845 5263 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46582:
name: "default.Table_fa2edebca3114f2b90a6b3f6a79f1699"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:54:17.620435 5265 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ea1051c6abd749979cb4e6c577078521 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:54:17.632769 5264 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d72b29e6aad6422384022c20cf87f343 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:54:17.633834 5263 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fa2edebca3114f2b90a6b3f6a79f1699 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:54:17.632756 5266 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2baacc33b03b46a9a286be0f0e9abe90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:54:17.648933 32301 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu with pid 5194
W20260810 07:54:17.703480 6154 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.139.124:41801)
W20260810 07:54:17.772294 6154 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.CreateTable -> {remote=127.31.139.124:41801, user_credentials={real_user=slave}} blocked reactor thread for 70799.2us
W20260810 07:54:17.915289 5904 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:17.925562 6187 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:17.930250 6187 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:17.932415 6187 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:17.933990 5050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
W20260810 07:54:17.947144 5125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260810 07:54:17.949595 6189 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:17.951704 6189 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
W20260810 07:54:17.968658 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:17.969326 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:17.970376 6187 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:54:17.973076 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918" is_pre_election: true
I20260810 07:54:17.973655 5158 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33049f5d320f4c6ca8d040c13fa09252 in term 1.
I20260810 07:54:17.977221 6189 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:17.978194 5008 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918; no voters:
I20260810 07:54:17.979207 6189 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260810 07:54:17.979522 5009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260810 07:54:17.979635 6189 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:17.980175 6189 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 1 FOLLOWER]: Advancing to term 2
W20260810 07:54:18.001871 5009 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:18.002277 6189 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:18.005722 6189 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 election: Requested vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:18.006631 5158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33049f5d320f4c6ca8d040c13fa09252" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c4cb37d55c894071b60fe8c11447e918"
I20260810 07:54:18.007174 5158 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:54:18.020815 5158 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33049f5d320f4c6ca8d040c13fa09252 in term 2.
W20260810 07:54:18.024221 5009 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:18.030310 5008 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918; no voters: a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:18.031467 6189 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:54:18.036485 6189 raft_consensus.cc:697] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 LEADER]: Becoming Leader. State: Replica: 33049f5d320f4c6ca8d040c13fa09252, State: Running, Role: LEADER
I20260810 07:54:18.037559 6189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:18.047276 6139 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
W20260810 07:54:18.057592 6139 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:18.059669 6196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33049f5d320f4c6ca8d040c13fa09252. Latest consensus state: current_term: 2 leader_uuid: "33049f5d320f4c6ca8d040c13fa09252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:18.061935 6196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:18.070070 6204 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:54:18.083411 6204 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:54:18.088220 6204 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:18.088634 6204 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:54:18.100006 6204 catalog_manager.cc:1540] Loading token signing keys...
W20260810 07:54:18.103945 5668 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:18.104692 6204 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Loaded TSK: 0
I20260810 07:54:18.106177 6204 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:54:18.110476 6204 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 07:54:18.114976 6204 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 07:54:18.150723 5030 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_fa2edebca3114f2b90a6b3f6a79f1699"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:18.154506 5028 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_d72b29e6aad6422384022c20cf87f343"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:18.172180 5032 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_ea1051c6abd749979cb4e6c577078521"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:54:18.172762 5030 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fa2edebca3114f2b90a6b3f6a79f1699 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:54:18.173930 5026 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:54:18.184374 5028 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d72b29e6aad6422384022c20cf87f343 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:54:18.218416 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:41801
--webserver_interface=127.31.139.124
--webserver_port=44291
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--hive_metastore_uris=thrift://127.0.0.1:43293
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260810 07:54:18.403596 5028 catalog_manager.cc:2200] created HMS catalog entry for table [id=2042666a41234cc3922f6ba5e13cd1ac]
W20260810 07:54:18.414891 5032 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ea1051c6abd749979cb4e6c577078521 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 07:54:18.419234 5026 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2baacc33b03b46a9a286be0f0e9abe90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:54:18.446380 5158 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Refusing update from remote peer 33049f5d320f4c6ca8d040c13fa09252: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260810 07:54:18.449676 6189 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260810 07:54:18.456758 5009 consensus_peers.cc:597] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 -> Peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Couldn't send request to peer a552a496f9e0418d97a2006bdef0af5a. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:54:18.502125 6187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33049f5d320f4c6ca8d040c13fa09252. Latest consensus state: current_term: 2 leader_uuid: "33049f5d320f4c6ca8d040c13fa09252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:18.503072 6187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:18.506558 5030 catalog_manager.cc:2200] created HMS catalog entry for table [id=ef3856c6f3044998ac5d4545c02ea0fd]
I20260810 07:54:18.506668 6208 mvcc.cc:204] Tried to move back new op lower bound from 7316883285746958336 to 7316883284153511936. Current Snapshot: MvccSnapshot[applied={T|T < 7316883285746958336}]
I20260810 07:54:18.510457 6210 mvcc.cc:204] Tried to move back new op lower bound from 7316883285746958336 to 7316883284153511936. Current Snapshot: MvccSnapshot[applied={T|T < 7316883285746958336}]
I20260810 07:54:18.537915 6196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "33049f5d320f4c6ca8d040c13fa09252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:18.538636 6196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:18.543282 6187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "33049f5d320f4c6ca8d040c13fa09252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:18.544081 6187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:18.561736 5030 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59766:
table { table_name: "default.Table_d72b29e6aad6422384022c20cf87f343" } new_table_name: "default.Table_58cc5e9b175943a39b188b4040251b54" modify_external_catalogs: true
I20260810 07:54:18.563143 5028 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59766:
table { table_name: "default.Table_fa2edebca3114f2b90a6b3f6a79f1699" } new_table_name: "default.Table_2421dcedd74043649cacf204a2130f13" modify_external_catalogs: true
I20260810 07:54:18.665285 5566 tablet_service.cc:1511] Processing CreateTablet for tablet 1f03ff5177244b3da7165d3200b945ea (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.667534 5566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03ff5177244b3da7165d3200b945ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.683699 5790 tablet_service.cc:1511] Processing CreateTablet for tablet 1f03ff5177244b3da7165d3200b945ea (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.686141 5790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03ff5177244b3da7165d3200b945ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.696810 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.698416 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b743816cbb4754925a5e1b767f0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.703838 5024 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f03ff5177244b3da7165d3200b945ea from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:18.707918 6229 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:18.725507 6229 tablet_bootstrap.cc:654] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:18.728916 6229 log.cc:826] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:18.729122 6028 tablet_service.cc:1511] Processing CreateTablet for tablet 1f03ff5177244b3da7165d3200b945ea (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.729718 6026 tablet_service.cc:1511] Processing CreateTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.731782 6028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03ff5177244b3da7165d3200b945ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.738078 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b743816cbb4754925a5e1b767f0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.746277 5023 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f03ff5177244b3da7165d3200b945ea from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:18.749253 5787 tablet_service.cc:1511] Processing CreateTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.750590 5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b743816cbb4754925a5e1b767f0ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.752682 5024 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:18.758569 5023 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_c1ffabac25f54e5eb1d0225b2ea9ac75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:18.764020 5024 catalog_manager.cc:6807] ReplaceTablet: tablet 1f03ff5177244b3da7165d3200b945ea deleted and replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e
I20260810 07:54:18.756947 6025 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f03ff5177244b3da7165d3200b945ea with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:18.769882 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f03ff5177244b3da7165d3200b945ea with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:18.770211 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f03ff5177244b3da7165d3200b945ea with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:18.787740 5024 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_17d9a8f6ddca4ecd861be1837d81ac39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:54:18.789505 5007 catalog_manager.cc:5017] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet 1f03ff5177244b3da7165d3200b945ea because the tablet was not found. No further retry: Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:18.792630 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 1f03ff5177244b3da7165d3200b945ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f03ff5177244b3da7165d3200b945ea already in progress: creating tablet
I20260810 07:54:18.799129 6229 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:18.799885 6229 ts_tablet_manager.cc:1403] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.093s user 0.028s sys 0.038s
W20260810 07:54:18.811492 5010 catalog_manager.cc:5017] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet 1f03ff5177244b3da7165d3200b945ea because the tablet was not found. No further retry: Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:18.822422 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.823726 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac854fe9bc34a72a29f00e98fb63a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.824503 5785 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.829152 6028 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:18.830194 5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac854fe9bc34a72a29f00e98fb63a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.830582 6028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac854fe9bc34a72a29f00e98fb63a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:18.852025 6237 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:18.856913 6238 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:18.876555 6238 tablet_bootstrap.cc:654] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:18.877923 6237 tablet_bootstrap.cc:654] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:18.879602 6238 log.cc:826] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:18.881553 6237 log.cc:826] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:18.892433 6238 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:18.893224 6238 ts_tablet_manager.cc:1403] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.037s user 0.017s sys 0.007s
I20260810 07:54:18.898375 6237 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:18.899065 6237 ts_tablet_manager.cc:1403] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.048s user 0.023s sys 0.000s
I20260810 07:54:18.925432 6238 raft_consensus.cc:359] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:18.926734 6238 raft_consensus.cc:385] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:18.927196 6238 raft_consensus.cc:740] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:18.928634 6238 consensus_queue.cc:260] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:18.939273 6237 raft_consensus.cc:359] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:18.940052 6238 ts_tablet_manager.cc:1434] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.046s user 0.030s sys 0.011s
I20260810 07:54:18.940490 6237 raft_consensus.cc:385] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:18.940941 6237 raft_consensus.cc:740] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:18.942233 6237 consensus_queue.cc:260] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:18.945752 5905 heartbeater.cc:499] Master 127.31.139.126:38803 was elected leader, sending a full tablet report...
I20260810 07:54:18.948525 6238 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:18.955905 6237 ts_tablet_manager.cc:1434] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.056s user 0.031s sys 0.009s
I20260810 07:54:18.960011 5020 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:18.963020 6238 tablet_bootstrap.cc:654] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:18.964370 6140 heartbeater.cc:499] Master 127.31.139.126:38803 was elected leader, sending a full tablet report...
I20260810 07:54:18.967557 6237 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:18.980996 6237 tablet_bootstrap.cc:654] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:18.984375 6238 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:18.984989 6238 ts_tablet_manager.cc:1403] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.006s
I20260810 07:54:18.988106 6238 raft_consensus.cc:359] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:18.988983 6238 raft_consensus.cc:385] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:18.989358 6238 raft_consensus.cc:740] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:18.990296 6238 consensus_queue.cc:260] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:18.993631 5016 master_service.cc:960] ReplaceTablet: received request to replace tablet 0ac854fe9bc34a72a29f00e98fb63a5e from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:18.996698 6238 ts_tablet_manager.cc:1434] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260810 07:54:19.001046 6238 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
W20260810 07:54:19.005694 6147 tablet.cc:2406] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:19.008199 6238 tablet_bootstrap.cc:654] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.013861 5020 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:18.988164 6229 raft_consensus.cc:359] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.015517 5020 catalog_manager.cc:6807] ReplaceTablet: tablet 21b743816cbb4754925a5e1b767f0ce7 deleted and replaced by tablet 2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.015524 6229 raft_consensus.cc:385] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.015990 6229 raft_consensus.cc:740] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.017062 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d02469c963e42b491c4be191ff12d46) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.017905 6028 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d02469c963e42b491c4be191ff12d46) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.017409 6229 consensus_queue.cc:260] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.018677 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d02469c963e42b491c4be191ff12d46) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.018999 6233 tablet_replica.cc:333] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:19.019613 6234 tablet_replica.cc:333] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:19.019908 6233 raft_consensus.cc:2243] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.020516 6233 raft_consensus.cc:2272] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
W20260810 07:54:19.021725 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 21b743816cbb4754925a5e1b767f0ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21b743816cbb4754925a5e1b767f0ce7 already in progress: creating tablet
I20260810 07:54:19.020467 6234 raft_consensus.cc:2243] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.022877 6234 raft_consensus.cc:2272] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.025504 6233 ts_tablet_manager.cc:1916] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.025920 6234 ts_tablet_manager.cc:1916] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:54:19.047560 5009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.139.124:41801, user_credentials={real_user=slave}} blocked reactor thread for 47574.2us
I20260810 07:54:19.052856 6234 ts_tablet_manager.cc:1929] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.053356 6234 log.cc:1199] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.056522 6234 ts_tablet_manager.cc:1950] T 21b743816cbb4754925a5e1b767f0ce7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
W20260810 07:54:19.057726 5911 tablet.cc:2406] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:19.059767 6028 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f03ff5177244b3da7165d3200b945ea with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.060164 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 21b743816cbb4754925a5e1b767f0ce7 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.062486 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f03ff5177244b3da7165d3200b945ea with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac854fe9bc34a72a29f00e98fb63a5e) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.063493 5673 heartbeater.cc:499] Master 127.31.139.126:38803 was elected leader, sending a full tablet report...
I20260810 07:54:19.065750 6234 tablet_replica.cc:333] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:19.066475 6234 raft_consensus.cc:2243] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.066982 6234 raft_consensus.cc:2272] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.069233 6234 ts_tablet_manager.cc:1916] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.071727 6229 ts_tablet_manager.cc:1434] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.271s user 0.028s sys 0.011s
I20260810 07:54:19.072290 6238 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:19.072846 6238 ts_tablet_manager.cc:1403] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.072s user 0.013s sys 0.022s
I20260810 07:54:19.073028 6229 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:19.076324 6238 raft_consensus.cc:359] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.077148 6238 raft_consensus.cc:385] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.077451 6238 raft_consensus.cc:740] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
W20260810 07:54:19.077883 5020 catalog_manager.cc:5552] Ignoring report from unknown tablet 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:19.078143 5020 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.078146 6238 consensus_queue.cc:260] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.080008 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b743816cbb4754925a5e1b767f0ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d02469c963e42b491c4be191ff12d46) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.086400 6237 tablet_bootstrap.cc:492] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.086858 6237 ts_tablet_manager.cc:1403] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.120s user 0.020s sys 0.018s
I20260810 07:54:19.088346 6238 ts_tablet_manager.cc:1434] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.015s user 0.004s sys 0.002s
I20260810 07:54:19.089798 6237 raft_consensus.cc:359] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.090487 6237 raft_consensus.cc:385] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.090764 6237 raft_consensus.cc:740] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.090919 5016 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet 0ac854fe9bc34a72a29f00e98fb63a5e
W20260810 07:54:19.092602 5017 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.095523 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e with delete_type TABLET_DATA_DELETED (replaced by tablet 23be15d143184d97aa18884dde07f3ec) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.095747 6028 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e with delete_type TABLET_DATA_DELETED (replaced by tablet 23be15d143184d97aa18884dde07f3ec) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.096479 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e with delete_type TABLET_DATA_DELETED (replaced by tablet 23be15d143184d97aa18884dde07f3ec) from {username='slave'} at 127.0.0.1:35386
W20260810 07:54:19.097595 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 0ac854fe9bc34a72a29f00e98fb63a5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ac854fe9bc34a72a29f00e98fb63a5e already in progress: creating tablet
I20260810 07:54:19.092643 5016 catalog_manager.cc:6807] ReplaceTablet: tablet 0ac854fe9bc34a72a29f00e98fb63a5e deleted and replaced by tablet 23be15d143184d97aa18884dde07f3ec
I20260810 07:54:19.102592 6237 consensus_queue.cc:260] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.105729 6234 ts_tablet_manager.cc:1929] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.106143 6234 log.cc:1199] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.106832 6234 ts_tablet_manager.cc:1950] T 1f03ff5177244b3da7165d3200b945ea P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:19.110050 6229 tablet_bootstrap.cc:654] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.110162 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 1f03ff5177244b3da7165d3200b945ea (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:19.110836 6234 tablet_replica.cc:333] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
W20260810 07:54:19.111320 5010 catalog_manager.cc:5017] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet 21b743816cbb4754925a5e1b767f0ce7 because the tablet was not found. No further retry: Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.111541 6234 raft_consensus.cc:2243] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.112044 6234 raft_consensus.cc:2272] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.115984 6234 ts_tablet_manager.cc:1916] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:54:19.120544 5016 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.139051 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02469c963e42b491c4be191ff12d46 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.140210 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02469c963e42b491c4be191ff12d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.147953 5785 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02469c963e42b491c4be191ff12d46 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.149137 5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02469c963e42b491c4be191ff12d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.151790 6028 tablet_service.cc:1511] Processing CreateTablet for tablet 2d02469c963e42b491c4be191ff12d46 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.153111 6028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d02469c963e42b491c4be191ff12d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.155323 5566 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e with delete_type TABLET_DATA_DELETED (replaced by tablet 23be15d143184d97aa18884dde07f3ec) from {username='slave'} at 127.0.0.1:53494
W20260810 07:54:19.157794 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 0ac854fe9bc34a72a29f00e98fb63a5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ac854fe9bc34a72a29f00e98fb63a5e already in progress: creating tablet
I20260810 07:54:19.158977 6229 tablet_bootstrap.cc:492] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.160044 6229 ts_tablet_manager.cc:1403] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.007s
I20260810 07:54:19.162956 6229 raft_consensus.cc:359] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.166347 6237 ts_tablet_manager.cc:1434] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.079s user 0.005s sys 0.000s
I20260810 07:54:19.163648 6229 raft_consensus.cc:385] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.167804 6229 raft_consensus.cc:740] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
W20260810 07:54:19.176927 5017 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.184830 6229 consensus_queue.cc:260] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.187224 6229 ts_tablet_manager.cc:1434] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.027s user 0.001s sys 0.004s
W20260810 07:54:19.193013 5020 catalog_manager.cc:5552] Ignoring report from unknown tablet 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.197855 6229 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:19.201243 6238 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
W20260810 07:54:19.202590 5020 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:19.202771 5017 catalog_manager.cc:5552] Ignoring report from unknown tablet 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.205199 6237 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:19.207933 6238 tablet_bootstrap.cc:654] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
W20260810 07:54:19.208401 5677 tablet.cc:2406] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:54:19.208911 6233 ts_tablet_manager.cc:1929] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.209383 6233 log.cc:1199] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.210362 6233 ts_tablet_manager.cc:1950] T 21b743816cbb4754925a5e1b767f0ce7 P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:19.218333 6229 tablet_bootstrap.cc:654] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.219252 6026 tablet_service.cc:1511] Processing CreateTablet for tablet 23be15d143184d97aa18884dde07f3ec (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.220417 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23be15d143184d97aa18884dde07f3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.220520 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 23be15d143184d97aa18884dde07f3ec (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.221833 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 21b743816cbb4754925a5e1b767f0ce7 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.226270 6237 tablet_bootstrap.cc:654] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.228801 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23be15d143184d97aa18884dde07f3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.229955 5785 tablet_service.cc:1511] Processing CreateTablet for tablet 23be15d143184d97aa18884dde07f3ec (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.230994 6233 tablet_replica.cc:333] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:19.231769 6233 raft_consensus.cc:2243] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.232409 6233 raft_consensus.cc:2272] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.234649 5020 master_service.cc:960] ReplaceTablet: received request to replace tablet 2d02469c963e42b491c4be191ff12d46 from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.235867 5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23be15d143184d97aa18884dde07f3ec. 1 dirs total, 0 dirs full, 0 dirs failed
07:54:19.236 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_ea1051c6abd749979cb4e6c577078521 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
I20260810 07:54:19.230305 6234 ts_tablet_manager.cc:1929] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.241403 6234 log.cc:1199] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/0ac854fe9bc34a72a29f00e98fb63a5e
I20260810 07:54:19.242177 6234 ts_tablet_manager.cc:1950] T 0ac854fe9bc34a72a29f00e98fb63a5e P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
W20260810 07:54:19.245196 5032 catalog_manager.cc:2196] Already present: failed to create HMS catalog entry for table [id=cc3947d3f1d24068ab4d56b3a11fe0ba]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_ea1051c6abd749979cb4e6c577078521 already exists)
I20260810 07:54:19.246132 5032 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59766 (request call id 20) took 1075 ms. Trace:
I20260810 07:54:19.246526 5032 rpcz_store.cc:276] 0810 07:54:18.171183 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:18.171762 (+ 579us) service_pool.cc:224] Handling call
0810 07:54:18.415458 (+243696us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:54:18.417424 (+ 1966us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:54:19.246043 (+828619us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":87812,"Hive Metastore.run_cpu_time_us":2170,"Hive Metastore.run_wall_time_us":738441}
I20260810 07:54:19.253818 5032 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_25f242ffd8e64a9b956f1b4a88b60eb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:19.256902 6238 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:19.256987 5020 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet 2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.257503 6238 ts_tablet_manager.cc:1403] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.002s
I20260810 07:54:19.258656 5020 catalog_manager.cc:6807] ReplaceTablet: tablet 2d02469c963e42b491c4be191ff12d46 deleted and replaced by tablet e3a9899a501644fabc0845b3d9b2fdee
I20260810 07:54:19.259354 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.261008 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 0ac854fe9bc34a72a29f00e98fb63a5e (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:19.263041 6025 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.265107 6233 ts_tablet_manager.cc:1916] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.271354 6229 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.271826 6229 ts_tablet_manager.cc:1403] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.006s
W20260810 07:54:19.272908 5010 catalog_manager.cc:5030] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet 2d02469c963e42b491c4be191ff12d46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d02469c963e42b491c4be191ff12d46 already in progress: creating tablet
I20260810 07:54:19.260319 6238 raft_consensus.cc:359] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.277179 6238 raft_consensus.cc:385] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.277701 6238 raft_consensus.cc:740] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.278590 6238 consensus_queue.cc:260] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.284436 6238 ts_tablet_manager.cc:1434] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260810 07:54:19.299444 6229 raft_consensus.cc:359] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.300164 6229 raft_consensus.cc:385] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.302327 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.305692 6234 tablet_replica.cc:333] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:19.306484 6234 raft_consensus.cc:2243] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.308269 6234 raft_consensus.cc:2272] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.312248 6238 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:19.313851 6026 tablet_service.cc:1511] Processing CreateTablet for tablet e3a9899a501644fabc0845b3d9b2fdee (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.314354 5787 tablet_service.cc:1511] Processing CreateTablet for tablet e3a9899a501644fabc0845b3d9b2fdee (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.315114 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a9899a501644fabc0845b3d9b2fdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.315442 5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a9899a501644fabc0845b3d9b2fdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.320775 6237 tablet_bootstrap.cc:492] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.321288 6237 ts_tablet_manager.cc:1403] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.116s user 0.008s sys 0.007s
I20260810 07:54:19.322995 6238 tablet_bootstrap.cc:654] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.324911 6234 ts_tablet_manager.cc:1916] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.325054 6237 raft_consensus.cc:359] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.325769 6237 raft_consensus.cc:385] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.326072 6237 raft_consensus.cc:740] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.337771 6237 consensus_queue.cc:260] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.343551 6233 ts_tablet_manager.cc:1929] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.343987 6233 log.cc:1199] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.358016 5020 master_service.cc:960] ReplaceTablet: received request to replace tablet 0ac854fe9bc34a72a29f00e98fb63a5e from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.358263 6237 ts_tablet_manager.cc:1434] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.037s user 0.007s sys 0.001s
I20260810 07:54:19.360918 5563 tablet_service.cc:1511] Processing CreateTablet for tablet e3a9899a501644fabc0845b3d9b2fdee (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.361763 6237 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:19.363947 5563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a9899a501644fabc0845b3d9b2fdee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.367951 6238 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:19.368422 6238 ts_tablet_manager.cc:1403] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.056s user 0.010s sys 0.001s
I20260810 07:54:19.371707 6238 raft_consensus.cc:359] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.372435 6238 raft_consensus.cc:385] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.373188 6233 ts_tablet_manager.cc:1950] T 1f03ff5177244b3da7165d3200b945ea P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:19.373484 5566 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.300464 6229 raft_consensus.cc:740] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.380931 6229 consensus_queue.cc:260] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.383929 6229 ts_tablet_manager.cc:1434] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.112s user 0.004s sys 0.004s
I20260810 07:54:19.387084 6229 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
W20260810 07:54:19.387950 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 2d02469c963e42b491c4be191ff12d46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d02469c963e42b491c4be191ff12d46 already in progress: creating tablet
I20260810 07:54:19.391060 6233 tablet_replica.cc:333] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:19.391759 6233 raft_consensus.cc:2243] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.392238 6233 raft_consensus.cc:2272] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.394171 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 1f03ff5177244b3da7165d3200b945ea (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:19.408164 5566 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ac854fe9bc34a72a29f00e98fb63a5e with delete_type TABLET_DATA_DELETED (replaced by tablet 23be15d143184d97aa18884dde07f3ec) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.409168 6235 tablet_replica.cc:333] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:19.410307 6237 tablet_bootstrap.cc:654] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.410311 6235 raft_consensus.cc:2243] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.410990 6235 raft_consensus.cc:2272] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.414677 6235 ts_tablet_manager.cc:1916] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.423949 6233 ts_tablet_manager.cc:1916] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.435227 6229 tablet_bootstrap.cc:654] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.372771 6238 raft_consensus.cc:740] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.437536 6238 consensus_queue.cc:260] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.440347 6238 ts_tablet_manager.cc:1434] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.072s user 0.007s sys 0.000s
I20260810 07:54:19.436882 6234 ts_tablet_manager.cc:1929] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.440927 6234 log.cc:1199] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.441730 6234 ts_tablet_manager.cc:1950] T 2d02469c963e42b491c4be191ff12d46 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:19.464474 6238 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:19.467105 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 2d02469c963e42b491c4be191ff12d46 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.474519 6237 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.474609 6233 ts_tablet_manager.cc:1929] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.474973 6233 log.cc:1199] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/0ac854fe9bc34a72a29f00e98fb63a5e
I20260810 07:54:19.475045 6237 ts_tablet_manager.cc:1403] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.008s
I20260810 07:54:19.475735 6233 ts_tablet_manager.cc:1950] T 0ac854fe9bc34a72a29f00e98fb63a5e P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:19.475762 5017 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f03ff5177244b3da7165d3200b945ea from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.486853 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 0ac854fe9bc34a72a29f00e98fb63a5e (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
W20260810 07:54:19.488066 5007 catalog_manager.cc:5030] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet 2d02469c963e42b491c4be191ff12d46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d02469c963e42b491c4be191ff12d46 already in progress: creating tablet
I20260810 07:54:19.491149 6258 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:19.491771 6258 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.495867 6237 raft_consensus.cc:359] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.496577 6258 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:19.502846 6229 tablet_bootstrap.cc:492] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.503420 6229 ts_tablet_manager.cc:1403] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.117s user 0.014s sys 0.000s
I20260810 07:54:19.506395 6229 raft_consensus.cc:359] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.508266 6229 raft_consensus.cc:385] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.508627 6229 raft_consensus.cc:740] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.509475 6229 consensus_queue.cc:260] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.507900 6235 ts_tablet_manager.cc:1929] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.515136 6258 raft_consensus.cc:493] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:19.517405 6235 log.cc:1199] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/0ac854fe9bc34a72a29f00e98fb63a5e
I20260810 07:54:19.518684 6237 raft_consensus.cc:385] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.519075 6237 raft_consensus.cc:740] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.518908 6235 ts_tablet_manager.cc:1950] T 0ac854fe9bc34a72a29f00e98fb63a5e P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:19.517313 6258 raft_consensus.cc:515] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.524900 6229 ts_tablet_manager.cc:1434] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260810 07:54:19.519901 6237 consensus_queue.cc:260] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.525580 6258 leader_election.cc:290] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:19.527349 6229 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:19.535457 6229 tablet_bootstrap.cc:654] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.541395 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:57642
W20260810 07:54:19.543256 5007 catalog_manager.cc:5030] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet 2d02469c963e42b491c4be191ff12d46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d02469c963e42b491c4be191ff12d46 already in progress: creating tablet
I20260810 07:54:19.547688 6238 tablet_bootstrap.cc:654] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.555390 6237 ts_tablet_manager.cc:1434] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.080s user 0.008s sys 0.000s
W20260810 07:54:19.558527 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:19.559540 5493 leader_election.cc:343] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.585377 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.586237 6233 tablet_replica.cc:333] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:19.586972 6233 raft_consensus.cc:2243] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.587494 6233 raft_consensus.cc:2272] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.594611 6233 ts_tablet_manager.cc:1916] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.595685 5017 master_service.cc:960] ReplaceTablet: received request to replace tablet e3a9899a501644fabc0845b3d9b2fdee from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.585588 6237 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
W20260810 07:54:19.606380 6207 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:19.607273 6207 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:19.608146 6207 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260810 07:54:19.611344 5007 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet 0ac854fe9bc34a72a29f00e98fb63a5e (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:19.616330 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d02469c963e42b491c4be191ff12d46 with delete_type TABLET_DATA_DELETED (replaced by tablet e3a9899a501644fabc0845b3d9b2fdee) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.617092 6235 tablet_replica.cc:333] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:19.617647 6237 tablet_bootstrap.cc:654] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.617841 6235 raft_consensus.cc:2243] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.618304 6235 raft_consensus.cc:2272] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.620679 6235 ts_tablet_manager.cc:1916] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.620968 5026 catalog_manager.cc:2200] created HMS catalog entry for table [id=32f58349ec154de5a8215c8a3b0a409e]
I20260810 07:54:19.631327 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d02469c963e42b491c4be191ff12d46" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce" is_pre_election: true
W20260810 07:54:19.633378 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:19.633885 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:19.635689 5017 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet e3a9899a501644fabc0845b3d9b2fdee
W20260810 07:54:19.636528 5489 leader_election.cc:343] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Illegal state: Tablet not RUNNING: STOPPED
I20260810 07:54:19.637030 5489 leader_election.cc:304] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
W20260810 07:54:19.637740 5489 raft_consensus.cc:2686] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260810 07:54:19.637956 6258 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:19.638302 5026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59766 (request call id 21) took 1466 ms. Trace:
I20260810 07:54:19.638706 5026 rpcz_store.cc:276] 0810 07:54:18.171586 (+ 0us) service_pool.cc:167] Inserting onto call queue
0810 07:54:18.173367 (+ 1781us) service_pool.cc:224] Handling call
0810 07:54:18.419966 (+246599us) catalog_manager.cc:2127] Acquired catalog manager lock
0810 07:54:18.424939 (+ 4973us) catalog_manager.cc:2173] Created new table and tablet info
0810 07:54:19.620915 (+1195976us) catalog_manager.cc:2199] Created new table in HMS catalog
0810 07:54:19.637451 (+ 16536us) catalog_manager.cc:2233] Wrote table and tablets to system table
0810 07:54:19.637864 (+ 413us) catalog_manager.cc:2267] Inserted table and tablets into CatalogManager maps
0810 07:54:19.638225 (+ 361us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0810 07:54:19.623349 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0810 07:54:19.623882 (+ 533us) write_op.cc:432] Acquiring schema lock in shared mode
0810 07:54:19.623913 (+ 31us) write_op.cc:435] Acquired schema lock
0810 07:54:19.623931 (+ 18us) tablet.cc:662] Decoding operations
0810 07:54:19.624290 (+ 359us) write_op.cc:620] Acquiring the partition lock for write op
0810 07:54:19.624343 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0810 07:54:19.624372 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0810 07:54:19.624662 (+ 290us) tablet.cc:701] Row locks acquired
0810 07:54:19.624681 (+ 19us) write_op.cc:260] PREPARE: finished
0810 07:54:19.624891 (+ 210us) write_op.cc:270] Start()
0810 07:54:19.624998 (+ 107us) write_op.cc:276] Timestamp: P: 1786348459624838 usec, L: 0
0810 07:54:19.625014 (+ 16us) op_driver.cc:348] REPLICATION: starting
0810 07:54:19.625712 (+ 698us) log.cc:844] Serialized 432 byte log entry
0810 07:54:19.635220 (+ 9508us) op_driver.cc:464] REPLICATION: finished
0810 07:54:19.635706 (+ 486us) write_op.cc:301] APPLY: starting
0810 07:54:19.635764 (+ 58us) tablet.cc:1367] starting BulkCheckPresence
0810 07:54:19.635904 (+ 140us) tablet.cc:1370] finished BulkCheckPresence
0810 07:54:19.636001 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0810 07:54:19.636540 (+ 539us) tablet.cc:1383] finished ApplyRowOperation cycle
0810 07:54:19.636686 (+ 146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0810 07:54:19.636747 (+ 61us) write_op.cc:312] APPLY: finished
0810 07:54:19.636961 (+ 214us) log.cc:844] Serialized 28 byte log entry
0810 07:54:19.637103 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:54:19.637179 (+ 76us) write_op.cc:454] Released schema lock
0810 07:54:19.637211 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0810 07:54:19.637435 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0810 07:54:19.637459 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":818899,"Hive Metastore.run_cpu_time_us":893,"Hive Metastore.run_wall_time_us":368373,"child_traces":[["op",{"apply.queue_time_us":326,"apply.run_cpu_time_us":1702,"apply.run_wall_time_us":1700,"num_ops":2,"prepare.queue_time_us":350,"prepare.run_cpu_time_us":2780,"prepare.run_wall_time_us":2781,"raft.queue_time_us":590,"raft.run_cpu_time_us":806,"raft.run_wall_time_us":805,"replication_time_us":10105}]]}
I20260810 07:54:19.647111 5026 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59766:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
I20260810 07:54:19.648213 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.648227 5026 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59766:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
I20260810 07:54:19.648816 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:35386
W20260810 07:54:19.650589 5010 catalog_manager.cc:5030] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet e3a9899a501644fabc0845b3d9b2fdee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3a9899a501644fabc0845b3d9b2fdee already in progress: creating tablet
I20260810 07:54:19.652894 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.653482 5017 catalog_manager.cc:6807] ReplaceTablet: tablet e3a9899a501644fabc0845b3d9b2fdee deleted and replaced by tablet fac4b0506a164a46b66a7328bef23d91
F20260810 07:54:19.653929 5403 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_ea1051c6abd749979cb4e6c577078521, message={"server":"","servicePrincipal":"","db":"default","table":"table_ea1051c6abd749979cb4e6c577078521","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_ea1051c6abd749979cb4e6c577078521\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348457},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_ea1051c6abd749979cb4e6c577078521\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_ea1051c6abd749979cb4e6c577078521\",\"kudu.table_id\":\"464515f666b04ceaa8ffab909babac70\",\"transient_lastDdlTime\":\"1786348457\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_d72b29e6aad6422384022c20cf87f343, message={"server":"","servicePrincipal":"","db":"default","table":"table_d72b29e6aad6422384022c20cf87f343","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_d72b29e6aad6422384022c20cf87f343\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348458},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_d72b29e6aad6422384022c20cf87f343\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_d72b29e6aad6422384022c20cf87f343\",\"kudu.table_id\":\"2042666a41234cc3922f6ba5e13cd1ac\",\"transient_lastDdlTime\":\"1786348458\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260810 07:54:19.656955 6229 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.657440 6238 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:19.657498 6229 ts_tablet_manager.cc:1403] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.130s user 0.024s sys 0.008s
*** Check failure stack trace: ***
I20260810 07:54:19.657927 6238 ts_tablet_manager.cc:1403] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.194s user 0.007s sys 0.005s
I20260810 07:54:19.660374 6229 raft_consensus.cc:359] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.661164 6229 raft_consensus.cc:385] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.660916 6238 raft_consensus.cc:359] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.661473 6229 raft_consensus.cc:740] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.661689 6238 raft_consensus.cc:385] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.661993 6238 raft_consensus.cc:740] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.662870 6238 consensus_queue.cc:260] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.666455 6229 consensus_queue.cc:260] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.674069 6237 tablet_bootstrap.cc:492] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.674641 6237 ts_tablet_manager.cc:1403] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.089s user 0.009s sys 0.006s
I20260810 07:54:19.676613 6229 ts_tablet_manager.cc:1434] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.019s user 0.004s sys 0.002s
I20260810 07:54:19.678042 6237 raft_consensus.cc:359] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.680051 6235 ts_tablet_manager.cc:1929] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.680462 6235 log.cc:1199] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.680516 6237 raft_consensus.cc:385] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.680996 6237 raft_consensus.cc:740] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.681203 6235 ts_tablet_manager.cc:1950] T 2d02469c963e42b491c4be191ff12d46 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:19.681870 6237 consensus_queue.cc:260] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.684366 6258 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:19.684760 5007 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet 2d02469c963e42b491c4be191ff12d46 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.684866 6258 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
W20260810 07:54:19.685662 5007 catalog_manager.cc:5030] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet e3a9899a501644fabc0845b3d9b2fdee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3a9899a501644fabc0845b3d9b2fdee already in progress: creating tablet
I20260810 07:54:19.686975 5020 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.686964 6258 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:19.683771 6229 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
W20260810 07:54:19.690656 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:19.691453 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:19.691890 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:19.693209 6258 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:19.696250 6229 tablet_bootstrap.cc:654] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.699167 6237 ts_tablet_manager.cc:1434] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.024s user 0.004s sys 0.002s
I20260810 07:54:19.708097 6229 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.710301 6237 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:19.713747 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:35386
W20260810 07:54:19.715600 5010 catalog_manager.cc:5030] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet e3a9899a501644fabc0845b3d9b2fdee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3a9899a501644fabc0845b3d9b2fdee already in progress: creating tablet
I20260810 07:54:19.708577 6229 ts_tablet_manager.cc:1403] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.007s
I20260810 07:54:19.720597 6238 ts_tablet_manager.cc:1434] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260810 07:54:19.720652 6229 raft_consensus.cc:359] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.721390 6229 raft_consensus.cc:385] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.721671 6229 raft_consensus.cc:740] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.724177 6229 consensus_queue.cc:260] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.732252 6229 ts_tablet_manager.cc:1434] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260810 07:54:19.734791 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:19.735627 6234 tablet_replica.cc:333] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:19.736546 6234 raft_consensus.cc:2243] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.737106 6234 raft_consensus.cc:2272] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.739408 6234 ts_tablet_manager.cc:1916] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.727430 6233 ts_tablet_manager.cc:1929] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.748562 6233 log.cc:1199] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/2d02469c963e42b491c4be191ff12d46
I20260810 07:54:19.749370 6233 ts_tablet_manager.cc:1950] T 2d02469c963e42b491c4be191ff12d46 P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:19.749469 6026 tablet_service.cc:1511] Processing CreateTablet for tablet e57cbd8ce3af440ea7c907f91a66753b (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.751806 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e57cbd8ce3af440ea7c907f91a66753b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.754544 5787 tablet_service.cc:1511] Processing CreateTablet for tablet e57cbd8ce3af440ea7c907f91a66753b (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.755311 5565 tablet_service.cc:1511] Processing CreateTablet for tablet e57cbd8ce3af440ea7c907f91a66753b (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.755756 5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e57cbd8ce3af440ea7c907f91a66753b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.756431 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e57cbd8ce3af440ea7c907f91a66753b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.756246 5563 tablet_service.cc:1511] Processing CreateTablet for tablet fac4b0506a164a46b66a7328bef23d91 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.757468 5563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4b0506a164a46b66a7328bef23d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.758912 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 2d02469c963e42b491c4be191ff12d46 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.765733 6237 tablet_bootstrap.cc:654] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
W20260810 07:54:19.766619 5007 catalog_manager.cc:5030] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet e3a9899a501644fabc0845b3d9b2fdee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3a9899a501644fabc0845b3d9b2fdee already in progress: creating tablet
I20260810 07:54:19.768520 5785 tablet_service.cc:1511] Processing CreateTablet for tablet fac4b0506a164a46b66a7328bef23d91 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.770186 5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4b0506a164a46b66a7328bef23d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.774385 6028 tablet_service.cc:1511] Processing CreateTablet for tablet fac4b0506a164a46b66a7328bef23d91 (DEFAULT_TABLE table=default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:19.778534 6028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4b0506a164a46b66a7328bef23d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:19.784025 5566 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.784893 6235 tablet_replica.cc:333] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:19.785707 6235 raft_consensus.cc:2243] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.786202 6235 raft_consensus.cc:2272] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus is shut down!
@ 0x7f37a2a87920 google::LogMessage::Flush() at ??:0
I20260810 07:54:19.788534 6235 ts_tablet_manager.cc:1916] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.793191 6229 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:19.818310 6229 tablet_bootstrap.cc:654] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.834834 6237 tablet_bootstrap.cc:492] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.835292 6237 ts_tablet_manager.cc:1403] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.125s user 0.008s sys 0.011s
I20260810 07:54:19.837657 6237 raft_consensus.cc:359] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.838135 6237 raft_consensus.cc:385] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.838306 6237 raft_consensus.cc:740] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.838903 6237 consensus_queue.cc:260] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.808650 6234 ts_tablet_manager.cc:1929] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
@ 0x7f37a2a8768f google::LogMessage::~LogMessage() at ??:0
I20260810 07:54:19.842284 6234 log.cc:1199] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/e3a9899a501644fabc0845b3d9b2fdee
I20260810 07:54:19.843502 6234 ts_tablet_manager.cc:1950] T e3a9899a501644fabc0845b3d9b2fdee P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:19.849704 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.850102 5020 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59766:
name: "default.Table_eef964168fae4ea3865d27a32a3af580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:19.850823 6237 ts_tablet_manager.cc:1434] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
W20260810 07:54:19.851423 5007 catalog_manager.cc:5030] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet e3a9899a501644fabc0845b3d9b2fdee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3a9899a501644fabc0845b3d9b2fdee already in progress: creating tablet
I20260810 07:54:19.862411 5016 master_service.cc:960] ReplaceTablet: received request to replace tablet e3a9899a501644fabc0845b3d9b2fdee from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:19.863483 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet e3a9899a501644fabc0845b3d9b2fdee with delete_type TABLET_DATA_DELETED (replaced by tablet fac4b0506a164a46b66a7328bef23d91) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:19.864410 6233 tablet_replica.cc:333] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:19.863476 6237 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:19.865224 6233 raft_consensus.cc:2243] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:19.865733 6233 raft_consensus.cc:2272] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:19.868363 6238 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:19.875219 6233 ts_tablet_manager.cc:1916] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:19.877023 6237 tablet_bootstrap.cc:654] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.881460 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet e3a9899a501644fabc0845b3d9b2fdee (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.882916 6229 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.883358 6229 ts_tablet_manager.cc:1403] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.090s user 0.009s sys 0.002s
I20260810 07:54:19.884302 6238 tablet_bootstrap.cc:654] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.885991 6229 raft_consensus.cc:359] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.886684 6229 raft_consensus.cc:385] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.886955 6229 raft_consensus.cc:740] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.887665 6229 consensus_queue.cc:260] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.854730 6235 ts_tablet_manager.cc:1929] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:19.889285 6237 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:19.889787 6237 ts_tablet_manager.cc:1403] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.012s
I20260810 07:54:19.888850 6235 log.cc:1199] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/e3a9899a501644fabc0845b3d9b2fdee
I20260810 07:54:19.890626 6235 ts_tablet_manager.cc:1950] T e3a9899a501644fabc0845b3d9b2fdee P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:19.892266 5007 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet e3a9899a501644fabc0845b3d9b2fdee (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
I20260810 07:54:19.892395 6237 raft_consensus.cc:359] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.893357 6237 raft_consensus.cc:385] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.893684 6237 raft_consensus.cc:740] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.894505 6237 consensus_queue.cc:260] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.896962 5016 master_service.cc:960] ReplaceTablet: received request to replace tablet e57cbd8ce3af440ea7c907f91a66753b from {username='slave'} at 127.0.0.1:59766
W20260810 07:54:19.901240 6207 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260810 07:54:19.901618 6207 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260810 07:54:19.902055 6207 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:54:19.902444 6207 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:19.902796 6207 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:54:19.903174 6207 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:54:19.912233 6229 ts_tablet_manager.cc:1434] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.029s user 0.006s sys 0.001s
@ 0x7f37adc07fbc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260810 07:54:19.917598 6229 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:19.919651 6248 raft_consensus.cc:493] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:19.920380 6238 tablet_bootstrap.cc:492] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:19.920816 6238 ts_tablet_manager.cc:1403] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.003s
I20260810 07:54:19.920447 6248 raft_consensus.cc:515] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.924176 6248 leader_election.cc:290] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:19.928484 6229 tablet_bootstrap.cc:654] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.923615 6238 raft_consensus.cc:359] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.938328 6229 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:19.938748 6229 ts_tablet_manager.cc:1403] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.005s
I20260810 07:54:19.937965 6238 raft_consensus.cc:385] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.941324 6229 raft_consensus.cc:359] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.941980 6229 raft_consensus.cc:385] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:19.942253 6229 raft_consensus.cc:740] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.942983 6229 consensus_queue.cc:260] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:19.940652 6238 raft_consensus.cc:740] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:19.945888 6229 ts_tablet_manager.cc:1434] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260810 07:54:19.946480 6258 raft_consensus.cc:493] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:19.946923 6258 raft_consensus.cc:515] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.947767 6238 consensus_queue.cc:260] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:19.948945 6258 leader_election.cc:290] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:19.949708 6238 ts_tablet_manager.cc:1434] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260810 07:54:19.950537 6238 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:19.956595 6237 ts_tablet_manager.cc:1434] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.066s user 0.003s sys 0.002s
I20260810 07:54:19.957319 6238 tablet_bootstrap.cc:654] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.958616 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23be15d143184d97aa18884dde07f3ec" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:19.959539 5815 raft_consensus.cc:2468] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:19.959836 6237 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:19.961442 5493 leader_election.cc:304] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:19.966100 6237 tablet_bootstrap.cc:654] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:19.968178 6258 raft_consensus.cc:2804] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:19.968565 6258 raft_consensus.cc:493] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:19.968935 6258 raft_consensus.cc:3060] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:19.977420 5016 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:19.981452 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet e57cbd8ce3af440ea7c907f91a66753b with delete_type TABLET_DATA_DELETED (replaced by tablet 3da3c276a91e403c962795ce2ad0b07c) from {username='slave'} at 127.0.0.1:35386
W20260810 07:54:19.983937 5010 catalog_manager.cc:5030] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): delete failed for tablet e57cbd8ce3af440ea7c907f91a66753b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e57cbd8ce3af440ea7c907f91a66753b already in progress: creating tablet
I20260810 07:54:19.995162 5016 catalog_manager.cc:6807] ReplaceTablet: tablet e57cbd8ce3af440ea7c907f91a66753b deleted and replaced by tablet 3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:19.996618 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet e57cbd8ce3af440ea7c907f91a66753b with delete_type TABLET_DATA_DELETED (replaced by tablet 3da3c276a91e403c962795ce2ad0b07c) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:19.999817 6235 tablet_replica.cc:333] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:20.000901 6235 raft_consensus.cc:2243] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:20.001437 6235 raft_consensus.cc:2272] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:20.002661 6233 ts_tablet_manager.cc:1929] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:20.003072 6233 log.cc:1199] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/e3a9899a501644fabc0845b3d9b2fdee
I20260810 07:54:20.003782 6233 ts_tablet_manager.cc:1950] T e3a9899a501644fabc0845b3d9b2fdee P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:20.003763 6235 ts_tablet_manager.cc:1916] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:20.005033 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet e57cbd8ce3af440ea7c907f91a66753b with delete_type TABLET_DATA_DELETED (replaced by tablet 3da3c276a91e403c962795ce2ad0b07c) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:20.012848 6258 raft_consensus.cc:515] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.014187 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23be15d143184d97aa18884dde07f3ec" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce" is_pre_election: true
I20260810 07:54:20.014882 6062 raft_consensus.cc:2468] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:20.019770 6235 ts_tablet_manager.cc:1929] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:20.020174 6235 log.cc:1199] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.020834 6235 ts_tablet_manager.cc:1950] T e57cbd8ce3af440ea7c907f91a66753b P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:20.024632 5007 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet e57cbd8ce3af440ea7c907f91a66753b (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
I20260810 07:54:20.029688 6026 tablet_service.cc:1511] Processing CreateTablet for tablet 3da3c276a91e403c962795ce2ad0b07c (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.030289 6258 leader_election.cc:290] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Requested vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:20.030874 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da3c276a91e403c962795ce2ad0b07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:20.031472 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet e3a9899a501644fabc0845b3d9b2fdee (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]) successfully deleted
W20260810 07:54:20.033376 5007 catalog_manager.cc:5030] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): delete failed for tablet e57cbd8ce3af440ea7c907f91a66753b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e57cbd8ce3af440ea7c907f91a66753b already in progress: creating tablet
I20260810 07:54:20.034461 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4b0506a164a46b66a7328bef23d91" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:20.034631 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23be15d143184d97aa18884dde07f3ec" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce"
I20260810 07:54:20.035117 5815 raft_consensus.cc:2468] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 0.
I20260810 07:54:20.035212 6062 raft_consensus.cc:3060] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.036899 6238 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:20.037295 6238 ts_tablet_manager.cc:1403] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.087s user 0.018s sys 0.027s
I20260810 07:54:20.037374 5787 tablet_service.cc:1511] Processing CreateTablet for tablet 3da3c276a91e403c962795ce2ad0b07c (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.038481 5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da3c276a91e403c962795ce2ad0b07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:20.039815 6238 raft_consensus.cc:359] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.040452 6238 raft_consensus.cc:385] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.040696 6238 raft_consensus.cc:740] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.041339 6238 consensus_queue.cc:260] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.041736 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23be15d143184d97aa18884dde07f3ec" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:20.042227 5815 raft_consensus.cc:3060] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.043237 6238 ts_tablet_manager.cc:1434] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260810 07:54:20.042616 6062 raft_consensus.cc:2468] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:20.048395 5563 tablet_service.cc:1558] Processing DeleteTablet for tablet e57cbd8ce3af440ea7c907f91a66753b with delete_type TABLET_DATA_DELETED (replaced by tablet 3da3c276a91e403c962795ce2ad0b07c) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:20.048559 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 3da3c276a91e403c962795ce2ad0b07c (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.050284 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da3c276a91e403c962795ce2ad0b07c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260810 07:54:20.050262 5007 catalog_manager.cc:5017] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet e57cbd8ce3af440ea7c907f91a66753b because the tablet was not found. No further retry: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.061537 5594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4b0506a164a46b66a7328bef23d91" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" is_pre_election: true
I20260810 07:54:20.062501 5594 raft_consensus.cc:2468] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 0.
I20260810 07:54:20.062995 5956 leader_election.cc:304] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5; no voters:
I20260810 07:54:20.063952 6248 raft_consensus.cc:2804] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:20.063870 5489 leader_election.cc:304] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:20.064352 6248 raft_consensus.cc:493] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:20.064742 6248 raft_consensus.cc:3060] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.071141 6258 raft_consensus.cc:2804] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:20.072039 5016 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:20.076967 6248 raft_consensus.cc:515] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.080521 6248 leader_election.cc:290] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Requested vote from peers d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:20.081094 6238 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:20.080886 5816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4b0506a164a46b66a7328bef23d91" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:20.081314 5594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4b0506a164a46b66a7328bef23d91" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90b26857dd7400b8f9dbd9e3216dca3"
I20260810 07:54:20.081555 5816 raft_consensus.cc:3060] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.081869 5594 raft_consensus.cc:3060] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.086678 6237 tablet_bootstrap.cc:492] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:20.087220 6237 ts_tablet_manager.cc:1403] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.128s user 0.013s sys 0.018s
I20260810 07:54:20.087368 6238 tablet_bootstrap.cc:654] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:20.088099 5816 raft_consensus.cc:2468] T fac4b0506a164a46b66a7328bef23d91 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 1.
I20260810 07:54:20.089375 5956 leader_election.cc:304] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5; no voters:
I20260810 07:54:20.090000 6237 raft_consensus.cc:359] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.090400 6248 raft_consensus.cc:2804] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:20.090651 6237 raft_consensus.cc:385] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.090953 6237 raft_consensus.cc:740] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.099618 6238 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:20.100082 6238 ts_tablet_manager.cc:1403] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.007s
I20260810 07:54:20.101558 6248 raft_consensus.cc:697] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [term 1 LEADER]: Becoming Leader. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Running, Role: LEADER
I20260810 07:54:20.102623 6238 raft_consensus.cc:359] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.103283 6238 raft_consensus.cc:385] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.102648 6248 consensus_queue.cc:237] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.103552 6238 raft_consensus.cc:740] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.104387 6238 consensus_queue.cc:260] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.108980 6238 ts_tablet_manager.cc:1434] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260810 07:54:20.109015 6237 consensus_queue.cc:260] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.111123 6237 ts_tablet_manager.cc:1434] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.023s user 0.004s sys 0.000s
W20260810 07:54:20.111738 5017 catalog_manager.cc:5552] Ignoring report from unknown tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.112020 5815 raft_consensus.cc:2468] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:20.120496 5594 raft_consensus.cc:2468] T fac4b0506a164a46b66a7328bef23d91 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 1.
W20260810 07:54:20.121145 5017 catalog_manager.cc:5552] Ignoring report from unknown tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.122926 5016 catalog_manager.cc:5719] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67). New cstate: current_term: 1 leader_uuid: "5c7100eee0ce4dae895ca9535696dfce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } health_report { overall_health: HEALTHY } } }
I20260810 07:54:20.132895 6258 raft_consensus.cc:697] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 LEADER]: Becoming Leader. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Running, Role: LEADER
I20260810 07:54:20.133956 6258 consensus_queue.cc:237] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
@ 0x7f37adc0777c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260810 07:54:20.143682 6229 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:20.149257 6229 tablet_bootstrap.cc:654] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:20.150512 6237 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
W20260810 07:54:20.153564 5016 catalog_manager.cc:5552] Ignoring report from unknown tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.157389 6237 tablet_bootstrap.cc:654] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:20.171727 6237 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:20.172150 6237 ts_tablet_manager.cc:1403] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20260810 07:54:20.176362 6237 raft_consensus.cc:359] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.177872 6237 raft_consensus.cc:385] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.178157 6237 raft_consensus.cc:740] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.178507 5017 catalog_manager.cc:5719] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65). New cstate: current_term: 1 leader_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } health_report { overall_health: UNKNOWN } } }
I20260810 07:54:20.179528 6229 tablet_bootstrap.cc:492] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:20.179975 6229 ts_tablet_manager.cc:1403] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.036s user 0.015s sys 0.007s
I20260810 07:54:20.182616 6229 raft_consensus.cc:359] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.183339 6229 raft_consensus.cc:385] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.183606 6229 raft_consensus.cc:740] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.184288 6229 consensus_queue.cc:260] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.178829 6237 consensus_queue.cc:260] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } }
I20260810 07:54:20.190583 6237 ts_tablet_manager.cc:1434] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.018s user 0.005s sys 0.002s
I20260810 07:54:20.204830 5018 master_service.cc:960] ReplaceTablet: received request to replace tablet 3da3c276a91e403c962795ce2ad0b07c from {username='slave'} at 127.0.0.1:59766
@ 0x7f37adc0b7f2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f37adc0b7aa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260810 07:54:20.237423 6229 ts_tablet_manager.cc:1434] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260810 07:54:20.246865 5018 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Sending DeleteTablet for 3 replicas of tablet 3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:20.249577 5563 tablet_service.cc:1558] Processing DeleteTablet for tablet 3da3c276a91e403c962795ce2ad0b07c with delete_type TABLET_DATA_DELETED (replaced by tablet 797f38a38e804b98b6e96247435d5876) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:20.249734 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet 3da3c276a91e403c962795ce2ad0b07c with delete_type TABLET_DATA_DELETED (replaced by tablet 797f38a38e804b98b6e96247435d5876) from {username='slave'} at 127.0.0.1:57642
I20260810 07:54:20.236879 6207 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:43293
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.124:41801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.139.124
--webserver_port=44291
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:20.250516 6235 tablet_replica.cc:333] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:20.250594 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 3da3c276a91e403c962795ce2ad0b07c with delete_type TABLET_DATA_DELETED (replaced by tablet 797f38a38e804b98b6e96247435d5876) from {username='slave'} at 127.0.0.1:53494
I20260810 07:54:20.251335 6233 tablet_replica.cc:333] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:20.251370 6235 raft_consensus.cc:2243] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:20.251873 6235 raft_consensus.cc:2272] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:20.252032 6233 raft_consensus.cc:2243] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:20.252391 6207 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:20.252532 6233 raft_consensus.cc:2272] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:20.254290 6235 ts_tablet_manager.cc:1916] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:20.255738 6233 ts_tablet_manager.cc:1916] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f37adc0b73a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260810 07:54:20.258246 6207 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260810 07:54:20.266750 5018 catalog_manager.cc:6807] ReplaceTablet: tablet 3da3c276a91e403c962795ce2ad0b07c deleted and replaced by tablet 797f38a38e804b98b6e96247435d5876
I20260810 07:54:20.267766 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet 3da3c276a91e403c962795ce2ad0b07c with delete_type TABLET_DATA_DELETED (replaced by tablet 797f38a38e804b98b6e96247435d5876) from {username='slave'} at 127.0.0.1:35386
I20260810 07:54:20.268802 6234 tablet_replica.cc:333] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:20.269570 6234 raft_consensus.cc:2243] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:20.270069 6234 raft_consensus.cc:2272] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:20.276417 6235 ts_tablet_manager.cc:1929] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:20.276893 6235 log.cc:1199] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:20.277629 6235 ts_tablet_manager.cc:1950] T 3da3c276a91e403c962795ce2ad0b07c P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:20.284348 5018 master_service.cc:960] ReplaceTablet: received request to replace tablet 3da3c276a91e403c962795ce2ad0b07c from {username='slave'} at 127.0.0.1:59766
I20260810 07:54:20.285127 6234 ts_tablet_manager.cc:1916] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f37adc0b702 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260810 07:54:20.289691 5007 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet 3da3c276a91e403c962795ce2ad0b07c (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
W20260810 07:54:20.290561 5007 catalog_manager.cc:5017] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): delete failed for tablet 3da3c276a91e403c962795ce2ad0b07c because the tablet was not found. No further retry: Not found: Tablet not found: 3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:20.310922 6234 ts_tablet_manager.cc:1929] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:20.311466 6234 log.cc:1199] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:20.312270 6234 ts_tablet_manager.cc:1950] T 3da3c276a91e403c962795ce2ad0b07c P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:20.315583 5010 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 3da3c276a91e403c962795ce2ad0b07c (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
I20260810 07:54:20.320990 5565 tablet_service.cc:1511] Processing CreateTablet for tablet 797f38a38e804b98b6e96247435d5876 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.322172 5565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797f38a38e804b98b6e96247435d5876. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f37adc0a9fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd8c5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd6fa std::__1::function<>::operator()() at ??:?
I20260810 07:54:20.347767 5787 tablet_service.cc:1511] Processing CreateTablet for tablet 797f38a38e804b98b6e96247435d5876 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.349066 5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797f38a38e804b98b6e96247435d5876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:20.355158 6233 ts_tablet_manager.cc:1929] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260810 07:54:20.355600 6233 log.cc:1199] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/3da3c276a91e403c962795ce2ad0b07c
I20260810 07:54:20.356278 6233 ts_tablet_manager.cc:1950] T 3da3c276a91e403c962795ce2ad0b07c P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:20.359131 5007 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 3da3c276a91e403c962795ce2ad0b07c (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
I20260810 07:54:20.362416 6026 tablet_service.cc:1511] Processing CreateTablet for tablet 797f38a38e804b98b6e96247435d5876 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:20.363497 6026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797f38a38e804b98b6e96247435d5876. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f37a4c711eb kudu::Thread::SuperviseThread() at ??:0
@ 0x4568e0 __tsan_thread_start_func at ??:?
W20260810 07:54:20.377693 6329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
@ 0x7f37a63d16db start_thread at ??:0
I20260810 07:54:20.407033 6229 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:20.409078 6237 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:20.414578 6229 tablet_bootstrap.cc:654] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:20.415541 6237 tablet_bootstrap.cc:654] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:20.425433 6238 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:20.430795 6238 tablet_bootstrap.cc:654] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
@ 0x7f37a172c61f clone at ??:0
I20260810 07:54:20.436978 6229 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:20.437521 6229 ts_tablet_manager.cc:1403] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.002s
I20260810 07:54:20.442164 6229 raft_consensus.cc:359] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.443037 6229 raft_consensus.cc:385] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.443351 6229 raft_consensus.cc:740] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.444126 6229 consensus_queue.cc:260] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.447444 6229 ts_tablet_manager.cc:1434] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260810 07:54:20.452149 6238 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:20.452582 6238 ts_tablet_manager.cc:1403] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.000s
I20260810 07:54:20.458451 6237 tablet_bootstrap.cc:492] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:20.458853 6237 ts_tablet_manager.cc:1403] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.050s user 0.008s sys 0.004s
I20260810 07:54:20.461388 6237 raft_consensus.cc:359] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.462000 6237 raft_consensus.cc:385] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.462230 6237 raft_consensus.cc:740] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.462926 6237 consensus_queue.cc:260] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.463752 6238 raft_consensus.cc:359] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.471328 6238 raft_consensus.cc:385] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:20.471611 6238 raft_consensus.cc:740] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:20.477015 6238 consensus_queue.cc:260] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.485783 6237 ts_tablet_manager.cc:1434] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260810 07:54:20.497314 6238 ts_tablet_manager.cc:1434] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.044s user 0.004s sys 0.001s
I20260810 07:54:20.594255 6248 raft_consensus.cc:493] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:20.594744 6248 raft_consensus.cc:515] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.596807 6248 leader_election.cc:290] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:20.598809 5594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797f38a38e804b98b6e96247435d5876" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" is_pre_election: true
I20260810 07:54:20.598799 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797f38a38e804b98b6e96247435d5876" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:20.599411 5594 raft_consensus.cc:2468] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 0.
I20260810 07:54:20.599396 5815 raft_consensus.cc:2468] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 0.
I20260810 07:54:20.600916 5951 leader_election.cc:304] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:20.602034 6248 raft_consensus.cc:2804] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:20.602375 6248 raft_consensus.cc:493] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:20.602662 6248 raft_consensus.cc:3060] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.609316 6248 raft_consensus.cc:515] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.612186 5594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797f38a38e804b98b6e96247435d5876" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90b26857dd7400b8f9dbd9e3216dca3"
I20260810 07:54:20.612772 5594 raft_consensus.cc:3060] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.620132 5594 raft_consensus.cc:2468] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 1.
I20260810 07:54:20.623521 5951 leader_election.cc:304] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:20.636624 6248 leader_election.cc:290] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Requested vote from peers d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:20.637219 6248 raft_consensus.cc:2804] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:20.637965 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797f38a38e804b98b6e96247435d5876" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:20.638513 5815 raft_consensus.cc:3060] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.637619 6248 raft_consensus.cc:697] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 1 LEADER]: Becoming Leader. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Running, Role: LEADER
I20260810 07:54:20.647310 6248 consensus_queue.cc:237] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:20.654179 5815 raft_consensus.cc:2468] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 1.
I20260810 07:54:20.719020 6248 raft_consensus.cc:493] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:20.719502 6248 raft_consensus.cc:515] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.726301 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e57cbd8ce3af440ea7c907f91a66753b" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:20.727203 5815 raft_consensus.cc:2468] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 0.
I20260810 07:54:20.729686 5956 leader_election.cc:304] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5; no voters:
I20260810 07:54:20.731106 6345 raft_consensus.cc:2804] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:20.731675 6345 raft_consensus.cc:493] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:20.732103 6345 raft_consensus.cc:3060] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:54:20.737324 5951 leader_election.cc:343] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.738266 6248 leader_election.cc:290] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:20.745219 6345 raft_consensus.cc:515] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.749529 5815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e57cbd8ce3af440ea7c907f91a66753b" candidate_uuid: "5c7100eee0ce4dae895ca9535696dfce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:20.750368 5815 raft_consensus.cc:3060] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:20.766680 5815 raft_consensus.cc:2468] T e57cbd8ce3af440ea7c907f91a66753b P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7100eee0ce4dae895ca9535696dfce in term 1.
I20260810 07:54:20.768800 5956 leader_election.cc:304] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5; no voters:
I20260810 07:54:20.770046 6248 raft_consensus.cc:2804] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election won for term 1
W20260810 07:54:20.780382 5951 leader_election.cc:343] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:20.782284 6345 leader_election.cc:290] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [CANDIDATE]: Term 1 election: Requested vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:20.793537 6248 raft_consensus.cc:697] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [term 1 LEADER]: Becoming Leader. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Running, Role: LEADER
I20260810 07:54:20.795222 6248 consensus_queue.cc:237] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:20.834990 6358 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33049f5d320f4c6ca8d040c13fa09252)
I20260810 07:54:20.836094 6358 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:20.845158 6358 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:20.861986 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:20.988180 5955 connection.cc:573] client connection to 127.31.139.126:38803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.988286 5712 connection.cc:573] server connection from 127.0.0.1:35386 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.988056 6154 connection.cc:573] client connection to 127.31.139.126:38803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.989272 5716 connection.cc:573] client connection to 127.31.139.126:38803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.989357 5492 connection.cc:573] client connection to 127.31.139.126:38803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.990249 5951 connection.cc:573] server connection from 127.0.0.1:57642 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:20.994415 5125 connection.cc:573] client connection to 127.31.139.126:38803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:21.000065 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
W20260810 07:54:21.001256 6154 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.31.139.126:38803)
W20260810 07:54:21.001659 6249 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:54:20.993189 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.31.139.126:38803 (local address 127.31.139.66:49391)
0810 07:54:20.998988 (+ 5799us) negotiation.cc:107] Waiting for socket to connect
0810 07:54:20.999057 (+ 69us) client_negotiation.cc:175] Beginning negotiation
0810 07:54:20.999642 (+ 585us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 07:54:21.000125 (+ 483us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":4808}
I20260810 07:54:21.003336 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
W20260810 07:54:21.004767 6154 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.31.139.126:38803)
I20260810 07:54:21.005434 6358 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260810 07:54:21.007735 6154 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.139.126:38803)
W20260810 07:54:21.010390 6154 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.31.139.126:38803)
I20260810 07:54:21.121805 6248 consensus_queue.cc:1048] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:54:21.122084 6367 consensus_queue.cc:1048] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:54:21.227494 6345 consensus_queue.cc:1048] T fac4b0506a164a46b66a7328bef23d91 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 07:54:21.303264 6367 consensus_queue.cc:1048] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:21.416601 6345 consensus_queue.cc:1048] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:21.459734 6248 consensus_queue.cc:1048] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:21.748268 6396 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33049f5d320f4c6ca8d040c13fa09252)
I20260810 07:54:21.749099 6396 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:21.752912 6396 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:21.768493 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
W20260810 07:54:21.770246 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:21.770922 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a
I20260810 07:54:21.772085 6396 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260810 07:54:21.837419 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:54:21.863775 6248 consensus_queue.cc:1048] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260810 07:54:21.715189 6328 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee60 after lost signal to thread 6207
W20260810 07:54:20.381285 6336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:22.243655 6207 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.879s user 0.592s sys 0.998s
W20260810 07:54:22.244081 6207 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.880s user 0.592s sys 0.998s
I20260810 07:54:22.253331 6207 server_base.cc:1061] running on GCE node
W20260810 07:54:22.253289 6411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:22.254771 6207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:22.257326 6207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:22.258751 6207 hybrid_clock.cc:648] HybridClock initialized: now 1786348462258700 us; error 34 us; skew 500 ppm
I20260810 07:54:22.259694 6207 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:22.266605 6207 webserver.cc:533] Webserver started at http://127.31.139.124:44291/ using document root <none> and password file <none>
I20260810 07:54:22.267971 6207 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:22.268297 6207 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:22.290122 6207 fs_manager.cc:714] Time spent opening directory manager: real 0.017s user 0.017s sys 0.001s
I20260810 07:54:22.296692 6423 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:22.298235 6207 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 07:54:22.298650 6207 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
uuid: "a552a496f9e0418d97a2006bdef0af5a"
format_stamp: "Formatted at 2026-08-10 07:54:07 on dist-test-slave-x5fp"
I20260810 07:54:22.301452 6207 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:22.377355 6207 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:22.378998 6207 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:22.379506 6207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:22.501992 6207 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.124:41801
I20260810 07:54:22.502050 6510 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.124:41801 every 8 connection(s)
I20260810 07:54:22.505575 6207 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-2/data/info.pb
I20260810 07:54:22.513595 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 6207
I20260810 07:54:22.521680 6511 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:54:22.535450 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Bootstrap starting.
I20260810 07:54:22.558128 6511 log.cc:826] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:22.579892 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:54:22.581039 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Bootstrap complete.
I20260810 07:54:22.603610 6511 raft_consensus.cc:359] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:22.606246 6511 raft_consensus.cc:740] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a552a496f9e0418d97a2006bdef0af5a, State: Initialized, Role: FOLLOWER
I20260810 07:54:22.607177 6511 consensus_queue.cc:260] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:22.610363 6517 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:22.611397 6517 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:22.612365 6511 sys_catalog.cc:565] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:54:22.658365 6540 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:22.658740 6540 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: loading cluster ID for follower catalog manager: success
I20260810 07:54:22.671671 6540 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: acquiring CA information for follower catalog manager: success
I20260810 07:54:22.645135 6207 master_runner.cc:187] Error getting master registration for 127.31.139.126:38803: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111),
I20260810 07:54:22.679286 6207 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:22.679704 6540 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:54:22.758936 6139 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:22.759477 6139 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
I20260810 07:54:22.768571 6453 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" instance_seqno: 1786348457427846) as {username='slave'} at 127.31.139.67:51065; Asking this server to re-register.
I20260810 07:54:22.770728 6139 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:22.774814 6453 ts_manager.cc:194] Registered new tserver with Master: 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:22.798761 5668 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:22.804529 6453 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" instance_seqno: 1786348452360200) as {username='slave'} at 127.31.139.65:60151; Asking this server to re-register.
I20260810 07:54:22.806151 5668 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:22.810091 6453 ts_manager.cc:194] Registered new tserver with Master: d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:22.893787 6550 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33049f5d320f4c6ca8d040c13fa09252)
I20260810 07:54:22.894361 6550 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:22.896338 6550 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:22.907392 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
I20260810 07:54:22.909941 6476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a" is_pre_election: true
I20260810 07:54:22.910756 6476 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4cb37d55c894071b60fe8c11447e918 in term 1.
I20260810 07:54:22.911908 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252
I20260810 07:54:22.912575 6550 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 07:54:22.912961 6550 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33049f5d320f4c6ca8d040c13fa09252)
I20260810 07:54:22.913352 6550 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:54:22.920603 6550 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:22.923012 6550 leader_election.cc:290] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 election: Requested vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
I20260810 07:54:22.924492 6476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4cb37d55c894071b60fe8c11447e918" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: false dest_uuid: "a552a496f9e0418d97a2006bdef0af5a"
I20260810 07:54:22.925122 6476 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 1 FOLLOWER]: Advancing to term 3
W20260810 07:54:22.927815 5125 leader_election.cc:336] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
I20260810 07:54:22.934190 6476 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4cb37d55c894071b60fe8c11447e918 in term 3.
I20260810 07:54:22.935328 5125 leader_election.cc:304] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918; no voters: 33049f5d320f4c6ca8d040c13fa09252
I20260810 07:54:22.937145 6550 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:54:22.941537 6550 raft_consensus.cc:697] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [term 3 LEADER]: Becoming Leader. State: Replica: c4cb37d55c894071b60fe8c11447e918, State: Running, Role: LEADER
I20260810 07:54:22.942613 6550 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:22.948616 6559 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4cb37d55c894071b60fe8c11447e918. Latest consensus state: current_term: 3 leader_uuid: "c4cb37d55c894071b60fe8c11447e918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:22.949434 6559 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:22.952901 6564 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:54:22.957562 5904 heartbeater.cc:344] Connected to a master server at 127.31.139.124:41801
I20260810 07:54:22.964265 6453 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" instance_seqno: 1786348455229866) as {username='slave'} at 127.31.139.66:44751; Asking this server to re-register.
I20260810 07:54:22.965027 6564 catalog_manager.cc:679] Loaded metadata for table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]
I20260810 07:54:22.966106 5904 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:22.967020 6564 catalog_manager.cc:679] Loaded metadata for table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]
I20260810 07:54:22.968711 6564 catalog_manager.cc:679] Loaded metadata for table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]
I20260810 07:54:22.970294 6453 ts_manager.cc:194] Registered new tserver with Master: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:22.975514 6564 tablet_loader.cc:96] loaded metadata for tablet 23be15d143184d97aa18884dde07f3ec (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac])
I20260810 07:54:22.977275 6564 tablet_loader.cc:96] loaded metadata for tablet 797f38a38e804b98b6e96247435d5876 (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e])
I20260810 07:54:22.978482 6564 tablet_loader.cc:96] loaded metadata for tablet fac4b0506a164a46b66a7328bef23d91 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd])
I20260810 07:54:22.979614 6564 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:54:22.983059 6564 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:22.983300 6564 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:54:22.987280 6564 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:54:22.991088 6564 catalog_manager.cc:6103] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Loaded TSK: 0
I20260810 07:54:22.992123 6564 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:54:22.995555 6564 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 07:54:22.998518 6564 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 07:54:23.010206 6476 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Refusing update from remote peer c4cb37d55c894071b60fe8c11447e918: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
W20260810 07:54:23.010428 5125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260810 07:54:23.011806 6559 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 4, Time since last communication: 0.000s
W20260810 07:54:23.013331 5125 consensus_peers.cc:597] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 -> Peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Couldn't send request to peer 33049f5d320f4c6ca8d040c13fa09252. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:54:23.048650 5145 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:57504
I20260810 07:54:23.057348 5133 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:57504
I20260810 07:54:23.080991 6517 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4cb37d55c894071b60fe8c11447e918. Latest consensus state: current_term: 3 leader_uuid: "c4cb37d55c894071b60fe8c11447e918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:23.086625 6570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c4cb37d55c894071b60fe8c11447e918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:23.087467 6570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:23.088078 6517 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:23.092847 6517 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c4cb37d55c894071b60fe8c11447e918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:23.093659 6517 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:23.108559 6550 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c4cb37d55c894071b60fe8c11447e918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:23.109428 6550 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918 [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:23.051445 5145 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57504:
name: "default.Table_25f242ffd8e64a9b956f1b4a88b60eb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:23.053711 5137 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57504:
name: "default.Table_eef964168fae4ea3865d27a32a3af580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:23.052125 5141 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57504:
name: "default.Table_17d9a8f6ddca4ecd861be1837d81ac39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:23.055274 5136 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57504:
name: "default.Table_c1ffabac25f54e5eb1d0225b2ea9ac75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:23.056903 5135 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57504:
table { table_name: "default.Table_fa2edebca3114f2b90a6b3f6a79f1699" } new_table_name: "default.Table_2421dcedd74043649cacf204a2130f13" modify_external_catalogs: true
I20260810 07:54:23.058473 5132 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57504:
table { table_name: "default.Table_d72b29e6aad6422384022c20cf87f343" } new_table_name: "default.Table_58cc5e9b175943a39b188b4040251b54" modify_external_catalogs: true
I20260810 07:54:23.050143 5143 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57504:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
I20260810 07:54:23.132877 5143 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57504:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
F20260810 07:54:23.148104 5464 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_ea1051c6abd749979cb4e6c577078521, message={"server":"","servicePrincipal":"","db":"default","table":"table_ea1051c6abd749979cb4e6c577078521","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_ea1051c6abd749979cb4e6c577078521\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348457},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_ea1051c6abd749979cb4e6c577078521\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_ea1051c6abd749979cb4e6c577078521\",\"kudu.table_id\":\"464515f666b04ceaa8ffab909babac70\",\"transient_lastDdlTime\":\"1786348457\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_d72b29e6aad6422384022c20cf87f343, message={"server":"","servicePrincipal":"","db":"default","table":"table_d72b29e6aad6422384022c20cf87f343","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_d72b29e6aad6422384022c20cf87f343\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348458},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_d72b29e6aad6422384022c20cf87f343\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_d72b29e6aad6422384022c20cf87f343\",\"kudu.table_id\":\"2042666a41234cc3922f6ba5e13cd1ac\",\"transient_lastDdlTime\":\"1786348458\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260810 07:54:23.161963 6567 mvcc.cc:204] Tried to move back new op lower bound from 7316883304475082752 to 7316883284153511936. Current Snapshot: MvccSnapshot[applied={T|T < 7316883304475082752 or (T in {7316883304475082752})}]
@ 0x7f59b8805920 google::LogMessage::Flush() at ??:0
@ 0x7f59b880568f google::LogMessage::~LogMessage() at ??:0
@ 0x7f59c3985fbc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f59c398577c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260810 07:54:23.269929 5147 master_service.cc:960] ReplaceTablet: received request to replace tablet 23be15d143184d97aa18884dde07f3ec from {username='slave'} at 127.0.0.1:57504
I20260810 07:54:23.270349 5133 master_service.cc:960] ReplaceTablet: received request to replace tablet 797f38a38e804b98b6e96247435d5876 from {username='slave'} at 127.0.0.1:57504
@ 0x7f59c39897f2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260810 07:54:23.302470 5133 catalog_manager.cc:6006] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Sending DeleteTablet for 3 replicas of tablet 797f38a38e804b98b6e96247435d5876
I20260810 07:54:23.307075 5133 catalog_manager.cc:6807] ReplaceTablet: tablet 797f38a38e804b98b6e96247435d5876 deleted and replaced by tablet b5ed3ebcc27c499aaaf8d33568b7d6f7
@ 0x7f59c39897aa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260810 07:54:23.335834 5147 catalog_manager.cc:6006] T 00000000000000000000000000000000 P c4cb37d55c894071b60fe8c11447e918: Sending DeleteTablet for 3 replicas of tablet 23be15d143184d97aa18884dde07f3ec
I20260810 07:54:23.337426 5147 catalog_manager.cc:6807] ReplaceTablet: tablet 23be15d143184d97aa18884dde07f3ec deleted and replaced by tablet eaac9de003b44d7288eb4194b2326cd0
@ 0x7f59c398973a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7f59c3989702 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7f59c39889fe std::__1::__function::__func<>::operator()() at ??:0
I20260810 07:54:23.435993 6028 tablet_service.cc:1558] Processing DeleteTablet for tablet 23be15d143184d97aa18884dde07f3ec with delete_type TABLET_DATA_DELETED (replaced by tablet eaac9de003b44d7288eb4194b2326cd0) from {username='slave'} at 127.0.0.1:54838
I20260810 07:54:23.436650 6026 tablet_service.cc:1558] Processing DeleteTablet for tablet 797f38a38e804b98b6e96247435d5876 with delete_type TABLET_DATA_DELETED (replaced by tablet b5ed3ebcc27c499aaaf8d33568b7d6f7) from {username='slave'} at 127.0.0.1:54838
I20260810 07:54:23.438345 6025 tablet_service.cc:1511] Processing CreateTablet for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.439929 6025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.441231 6024 tablet_service.cc:1511] Processing CreateTablet for tablet eaac9de003b44d7288eb4194b2326cd0 (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.442338 6024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaac9de003b44d7288eb4194b2326cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.455634 6604 tablet_replica.cc:333] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
I20260810 07:54:23.459640 5563 tablet_service.cc:1558] Processing DeleteTablet for tablet 23be15d143184d97aa18884dde07f3ec with delete_type TABLET_DATA_DELETED (replaced by tablet eaac9de003b44d7288eb4194b2326cd0) from {username='slave'} at 127.0.0.1:45714
I20260810 07:54:23.460790 6604 raft_consensus.cc:2243] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:23.461119 5785 tablet_service.cc:1558] Processing DeleteTablet for tablet 23be15d143184d97aa18884dde07f3ec with delete_type TABLET_DATA_DELETED (replaced by tablet eaac9de003b44d7288eb4194b2326cd0) from {username='slave'} at 127.0.0.1:51288
I20260810 07:54:23.461519 6604 raft_consensus.cc:2272] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.461506 5566 tablet_service.cc:1511] Processing CreateTablet for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.463124 5566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.462841 5790 tablet_service.cc:1511] Processing CreateTablet for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7 (DEFAULT_TABLE table=default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.463855 6604 ts_tablet_manager.cc:1916] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.464238 5790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.465633 5784 tablet_service.cc:1511] Processing CreateTablet for tablet eaac9de003b44d7288eb4194b2326cd0 (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.465829 5561 tablet_service.cc:1511] Processing CreateTablet for tablet eaac9de003b44d7288eb4194b2326cd0 (DEFAULT_TABLE table=default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:54:23.466795 5784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaac9de003b44d7288eb4194b2326cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.467151 5561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaac9de003b44d7288eb4194b2326cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:54:23.468968 5565 tablet_service.cc:1558] Processing DeleteTablet for tablet 797f38a38e804b98b6e96247435d5876 with delete_type TABLET_DATA_DELETED (replaced by tablet b5ed3ebcc27c499aaaf8d33568b7d6f7) from {username='slave'} at 127.0.0.1:45714
I20260810 07:54:23.460491 5787 tablet_service.cc:1558] Processing DeleteTablet for tablet 797f38a38e804b98b6e96247435d5876 with delete_type TABLET_DATA_DELETED (replaced by tablet b5ed3ebcc27c499aaaf8d33568b7d6f7) from {username='slave'} at 127.0.0.1:51288
@ 0x5bd8c5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd6fa std::__1::function<>::operator()() at ??:?
I20260810 07:54:23.510597 6607 tablet_replica.cc:333] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:23.511478 6607 raft_consensus.cc:2243] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:23.512147 6607 raft_consensus.cc:2272] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.514091 6606 tablet_replica.cc:333] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:23.514932 6606 raft_consensus.cc:2243] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:54:23.516007 6606 raft_consensus.cc:2272] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.518113 6606 ts_tablet_manager.cc:1916] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.526932 6607 ts_tablet_manager.cc:1916] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.528363 5147 master_service.cc:960] ReplaceTablet: received request to replace tablet 797f38a38e804b98b6e96247435d5876 from {username='slave'} at 127.0.0.1:57504
I20260810 07:54:23.533824 6605 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:23.542032 6608 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:23.551152 5147 master_service.cc:960] ReplaceTablet: received request to replace tablet 23be15d143184d97aa18884dde07f3ec from {username='slave'} at 127.0.0.1:57504
I20260810 07:54:23.533962 6604 ts_tablet_manager.cc:1929] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.557235 6604 log.cc:1199] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/23be15d143184d97aa18884dde07f3ec
I20260810 07:54:23.558252 6604 ts_tablet_manager.cc:1950] T 23be15d143184d97aa18884dde07f3ec P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:23.561914 6609 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
I20260810 07:54:23.564440 6605 tablet_bootstrap.cc:654] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:23.566018 6608 tablet_bootstrap.cc:654] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:23.566658 5122 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 23be15d143184d97aa18884dde07f3ec (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:23.568367 6604 tablet_replica.cc:333] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: stopping tablet replica
@ 0x7f59ba9ef1eb kudu::Thread::SuperviseThread() at ??:0
@ 0x4568e0 __tsan_thread_start_func at ??:?
I20260810 07:54:23.571848 6604 raft_consensus.cc:2243] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:54:23.572937 6607 ts_tablet_manager.cc:1929] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.573415 6607 log.cc:1199] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/23be15d143184d97aa18884dde07f3ec
I20260810 07:54:23.574507 6607 ts_tablet_manager.cc:1950] T 23be15d143184d97aa18884dde07f3ec P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:23.579174 5127 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 23be15d143184d97aa18884dde07f3ec (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:23.580302 6607 tablet_replica.cc:333] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: stopping tablet replica
I20260810 07:54:23.581055 6607 raft_consensus.cc:2243] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:23.581590 6607 raft_consensus.cc:2272] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.583596 6607 ts_tablet_manager.cc:1916] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.584529 6606 ts_tablet_manager.cc:1929] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.584930 6606 log.cc:1199] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/23be15d143184d97aa18884dde07f3ec
I20260810 07:54:23.585708 6606 ts_tablet_manager.cc:1950] T 23be15d143184d97aa18884dde07f3ec P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:23.573215 6604 raft_consensus.cc:2272] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.589294 5122 catalog_manager.cc:5050] TS d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): tablet 23be15d143184d97aa18884dde07f3ec (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]) successfully deleted
I20260810 07:54:23.590251 6606 tablet_replica.cc:333] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: stopping tablet replica
I20260810 07:54:23.590809 6604 ts_tablet_manager.cc:1916] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.593693 6605 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:23.594103 6605 ts_tablet_manager.cc:1403] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.061s user 0.017s sys 0.000s
I20260810 07:54:23.597064 6609 tablet_bootstrap.cc:654] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:23.597203 6605 raft_consensus.cc:359] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.590970 6606 raft_consensus.cc:2243] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:54:23.599256 6606 raft_consensus.cc:2272] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:54:23.601943 6606 ts_tablet_manager.cc:1916] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:54:23.606846 6608 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:23.607337 6608 ts_tablet_manager.cc:1403] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.066s user 0.011s sys 0.000s
I20260810 07:54:23.609066 6607 ts_tablet_manager.cc:1929] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.609598 6607 log.cc:1199] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-1/wal/wals/797f38a38e804b98b6e96247435d5876
I20260810 07:54:23.611102 6607 ts_tablet_manager.cc:1950] T 797f38a38e804b98b6e96247435d5876 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Deleting consensus metadata
I20260810 07:54:23.598304 6605 raft_consensus.cc:385] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.611598 6605 raft_consensus.cc:740] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.612579 6605 consensus_queue.cc:260] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.614096 6138 heartbeater.cc:499] Master 127.31.139.125:41987 was elected leader, sending a full tablet report...
I20260810 07:54:23.610615 6608 raft_consensus.cc:359] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.615402 6608 raft_consensus.cc:385] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.615744 6608 raft_consensus.cc:740] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.616899 6608 consensus_queue.cc:260] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.619073 5127 catalog_manager.cc:5050] TS 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): tablet 797f38a38e804b98b6e96247435d5876 (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
I20260810 07:54:23.626717 6609 tablet_bootstrap.cc:492] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:23.627192 6609 ts_tablet_manager.cc:1403] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.066s user 0.000s sys 0.011s
I20260810 07:54:23.628441 6605 ts_tablet_manager.cc:1434] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.034s user 0.004s sys 0.003s
W20260810 07:54:23.630167 5147 catalog_manager.cc:5552] Ignoring report from unknown tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:23.630054 6609 raft_consensus.cc:359] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.630746 6609 raft_consensus.cc:385] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.631031 6609 raft_consensus.cc:740] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.631754 6609 consensus_queue.cc:260] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.635612 5666 heartbeater.cc:499] Master 127.31.139.125:41987 was elected leader, sending a full tablet report...
I20260810 07:54:23.638082 6609 ts_tablet_manager.cc:1434] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260810 07:54:23.639021 6609 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3: Bootstrap starting.
W20260810 07:54:23.645056 5133 catalog_manager.cc:5552] Ignoring report from unknown tablet 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:23.645562 5133 catalog_manager.cc:5552] Ignoring report from unknown tablet 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:23.658576 6605 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce: Bootstrap starting.
I20260810 07:54:23.667057 6604 ts_tablet_manager.cc:1929] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.667477 6604 log.cc:1199] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-2/wal/wals/797f38a38e804b98b6e96247435d5876
I20260810 07:54:23.668231 6604 ts_tablet_manager.cc:1950] T 797f38a38e804b98b6e96247435d5876 P 5c7100eee0ce4dae895ca9535696dfce: Deleting consensus metadata
I20260810 07:54:23.674548 6605 tablet_bootstrap.cc:654] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce: Neither blocks nor log segments found. Creating new log.
@ 0x7f59bc14f6db start_thread at ??:0
I20260810 07:54:23.677330 6609 tablet_bootstrap.cc:654] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:23.678545 5902 heartbeater.cc:499] Master 127.31.139.125:41987 was elected leader, sending a full tablet report...
I20260810 07:54:23.681111 6608 ts_tablet_manager.cc:1434] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.073s user 0.021s sys 0.036s
I20260810 07:54:23.682098 6608 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Bootstrap starting.
I20260810 07:54:23.687544 5122 catalog_manager.cc:5050] TS 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): tablet 797f38a38e804b98b6e96247435d5876 (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]) successfully deleted
W20260810 07:54:23.694545 5147 catalog_manager.cc:5552] Ignoring report from unknown tablet e57cbd8ce3af440ea7c907f91a66753b
I20260810 07:54:23.697244 6608 tablet_bootstrap.cc:654] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:54:23.702759 6609 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3: No bootstrap required, opened a new log
I20260810 07:54:23.703197 6609 ts_tablet_manager.cc:1403] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent bootstrapping tablet: real 0.064s user 0.010s sys 0.000s
I20260810 07:54:23.705997 6609 raft_consensus.cc:359] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.706671 6609 raft_consensus.cc:385] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.706959 6609 raft_consensus.cc:740] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.707633 6609 consensus_queue.cc:260] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.711396 6608 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: No bootstrap required, opened a new log
I20260810 07:54:23.711808 6608 ts_tablet_manager.cc:1403] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.005s
I20260810 07:54:23.714938 6608 raft_consensus.cc:359] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.715628 6608 raft_consensus.cc:385] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.715898 6608 raft_consensus.cc:740] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.716594 6608 consensus_queue.cc:260] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.718797 6608 ts_tablet_manager.cc:1434] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260810 07:54:23.729948 6609 ts_tablet_manager.cc:1434] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3: Time spent starting tablet: real 0.026s user 0.000s sys 0.007s
@ 0x7f59b74aa61f clone at ??:0
I20260810 07:54:23.746078 6606 ts_tablet_manager.cc:1929] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260810 07:54:23.746472 6606 log.cc:1199] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting WAL directory at /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/ts-0/wal/wals/797f38a38e804b98b6e96247435d5876
I20260810 07:54:23.747241 6606 ts_tablet_manager.cc:1950] T 797f38a38e804b98b6e96247435d5876 P d90b26857dd7400b8f9dbd9e3216dca3: Deleting consensus metadata
I20260810 07:54:23.750501 6605 tablet_bootstrap.cc:492] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce: No bootstrap required, opened a new log
I20260810 07:54:23.750949 6605 ts_tablet_manager.cc:1403] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.003s
I20260810 07:54:23.753726 6605 raft_consensus.cc:359] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.754489 6605 raft_consensus.cc:385] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:54:23.754783 6605 raft_consensus.cc:740] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7100eee0ce4dae895ca9535696dfce, State: Initialized, Role: FOLLOWER
I20260810 07:54:23.757030 6605 consensus_queue.cc:260] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.757946 6580 raft_consensus.cc:493] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:23.758359 6580 raft_consensus.cc:515] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.760413 6580 leader_election.cc:290] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:23.761952 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaac9de003b44d7288eb4194b2326cd0" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce" is_pre_election: true
I20260810 07:54:23.762532 6062 raft_consensus.cc:2468] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:23.764938 5489 leader_election.cc:304] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:23.767406 6605 ts_tablet_manager.cc:1434] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce: Time spent starting tablet: real 0.016s user 0.003s sys 0.004s
I20260810 07:54:23.771252 5816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaac9de003b44d7288eb4194b2326cd0" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:23.771956 5816 raft_consensus.cc:2468] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:23.777612 6580 raft_consensus.cc:2804] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:23.778152 6580 raft_consensus.cc:493] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:23.778532 6580 raft_consensus.cc:3060] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:23.785389 6580 raft_consensus.cc:515] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.788305 5816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaac9de003b44d7288eb4194b2326cd0" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:23.788857 5816 raft_consensus.cc:3060] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:23.796979 5816 raft_consensus.cc:2468] T eaac9de003b44d7288eb4194b2326cd0 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:23.798374 5493 leader_election.cc:304] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:23.801378 6627 raft_consensus.cc:2804] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:23.803036 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaac9de003b44d7288eb4194b2326cd0" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce"
I20260810 07:54:23.803572 6062 raft_consensus.cc:3060] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:23.810139 6062 raft_consensus.cc:2468] T eaac9de003b44d7288eb4194b2326cd0 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:23.811409 6580 leader_election.cc:290] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Requested vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:23.812414 6627 raft_consensus.cc:697] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 LEADER]: Becoming Leader. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Running, Role: LEADER
I20260810 07:54:23.813333 6627 consensus_queue.cc:237] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.872576 6627 raft_consensus.cc:493] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:23.873155 6627 raft_consensus.cc:515] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.875665 6627 leader_election.cc:290] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:23.876528 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ed3ebcc27c499aaaf8d33568b7d6f7" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce" is_pre_election: true
I20260810 07:54:23.876932 5816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ed3ebcc27c499aaaf8d33568b7d6f7" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" is_pre_election: true
I20260810 07:54:23.877241 6062 raft_consensus.cc:2468] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:23.877566 5816 raft_consensus.cc:2468] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 0.
I20260810 07:54:23.878444 5489 leader_election.cc:304] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:23.879300 6627 raft_consensus.cc:2804] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:54:23.879623 6627 raft_consensus.cc:493] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:54:23.879930 6627 raft_consensus.cc:3060] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:54:23.879505 6634 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
I20260810 07:54:23.881296 6634 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:23.887487 6634 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
I20260810 07:54:23.891671 6627 raft_consensus.cc:515] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.894608 6062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ed3ebcc27c499aaaf8d33568b7d6f7" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7100eee0ce4dae895ca9535696dfce"
I20260810 07:54:23.895185 6062 raft_consensus.cc:3060] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:54:23.898698 6429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
I20260810 07:54:23.903177 6062 raft_consensus.cc:2468] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 5c7100eee0ce4dae895ca9535696dfce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:23.904574 5489 leader_election.cc:304] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7100eee0ce4dae895ca9535696dfce, d90b26857dd7400b8f9dbd9e3216dca3; no voters:
I20260810 07:54:23.905490 6580 raft_consensus.cc:2804] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:54:23.905954 6580 raft_consensus.cc:697] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 1 LEADER]: Becoming Leader. State: Replica: d90b26857dd7400b8f9dbd9e3216dca3, State: Running, Role: LEADER
I20260810 07:54:23.906728 6580 consensus_queue.cc:237] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:23.913657 6629 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 07:54:23.914350 6429 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
I20260810 07:54:23.914207 6629 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:23.916334 6629 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
I20260810 07:54:23.916601 6627 leader_election.cc:290] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 election: Requested vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:54:23.919857 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:23.919834 5813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ed3ebcc27c499aaaf8d33568b7d6f7" candidate_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5"
I20260810 07:54:23.920763 5813 raft_consensus.cc:3060] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:54:23.923758 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:23.924194 5493 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:23.930126 5813 raft_consensus.cc:2468] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d90b26857dd7400b8f9dbd9e3216dca3 in term 1.
I20260810 07:54:23.931898 6580 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:54:24.073494 5953 connection.cc:573] client connection to 127.31.139.125:41987 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:24.074026 6153 connection.cc:573] client connection to 127.31.139.125:41987 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:54:24.079160 6580 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:24.079826 6580 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
W20260810 07:54:24.082123 5491 connection.cc:573] server connection from 127.0.0.1:45714 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:24.076683 6153 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.31.139.125:41987)
W20260810 07:54:24.084380 6153 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.139.125:41987)
W20260810 07:54:24.085634 6153 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.31.139.125:41987)
W20260810 07:54:24.086615 6636 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:54:23.885394 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.31.139.125:41987 (local address 127.0.0.1:47598)
0810 07:54:23.888947 (+ 3553us) negotiation.cc:107] Waiting for socket to connect
0810 07:54:23.889004 (+ 57us) client_negotiation.cc:175] Beginning negotiation
0810 07:54:23.889351 (+ 347us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0810 07:54:24.084504 (+195153us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2922,"thread_start_us":1451,"threads_started":1}
W20260810 07:54:24.087709 6153 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.31.139.125:41987)
W20260810 07:54:24.094164 6545 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:54:24.075896 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.31.139.125:41987 (local address 127.31.139.67:33433)
0810 07:54:24.077104 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0810 07:54:24.077335 (+ 231us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":449}
W20260810 07:54:24.096421 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:24.100854 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:24.101346 6427 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a; no voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:24.099227 6580 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:24.102151 6634 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260810 07:54:24.102248 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:24.102694 5489 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:24.103535 6580 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:24.590762 6580 consensus_queue.cc:1048] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:24.616670 6580 consensus_queue.cc:1048] T eaac9de003b44d7288eb4194b2326cd0 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:24.814692 6667 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
I20260810 07:54:24.815290 6667 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:24.817893 6667 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
W20260810 07:54:24.825651 6429 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
W20260810 07:54:24.826478 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:24.826895 6427 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a; no voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:24.827623 6667 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:54:24.983129 6649 consensus_queue.cc:1048] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:25.005782 6649 consensus_queue.cc:1048] T b5ed3ebcc27c499aaaf8d33568b7d6f7 P d90b26857dd7400b8f9dbd9e3216dca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:54:25.338146 6678 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
I20260810 07:54:25.338572 6678 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:25.340045 6678 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
W20260810 07:54:25.345942 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:25.346462 6429 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
I20260810 07:54:25.346810 6429 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a; no voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:25.347460 6678 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:54:25.802106 32301 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
/tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38803
--webserver_interface=127.31.139.126
--webserver_port=37953
--builtin_ntp_servers=127.31.139.84:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--hive_metastore_uris=thrift://127.0.0.1:43293
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260810 07:54:26.166805 6699 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:54:26.167480 6699 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:54:26.167955 6699 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260810 07:54:26.205009 6699 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260810 07:54:26.205266 6699 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260810 07:54:26.205478 6699 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:54:26.205678 6699 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:54:26.205878 6699 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:54:26.206086 6699 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:54:26.231099 6710 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
I20260810 07:54:26.231575 6710 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:26.233316 6710 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
W20260810 07:54:26.240797 6429 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803): Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111)
W20260810 07:54:26.241379 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:26.241789 6427 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a552a496f9e0418d97a2006bdef0af5a; no voters: 33049f5d320f4c6ca8d040c13fa09252, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:26.242421 6710 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260810 07:54:26.251015 6699 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.139.84:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:43293
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.139.126:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.139.126
--webserver_port=37953
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 10 Aug 2026 07:43:25 UTC on e7f111948823
build id 13271
TSAN enabled
I20260810 07:54:26.252516 6699 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:54:26.254308 6699 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260810 07:54:26.267997 6717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:54:27.723930 6207 master_runner.cc:187] Error getting master registration for 127.31.139.126:38803: Network error: Client connection negotiation failed: client connection to 127.31.139.126:38803: connect: Connection refused (error 111),
I20260810 07:54:27.724361 6207 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:54:27.672122 6716 debug-util.cc:398] Leaking SignalData structure 0x7b080002ee60 after lost signal to thread 6699
W20260810 07:54:28.081394 6699 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.813s user 0.622s sys 1.135s
W20260810 07:54:26.268992 6718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:28.081813 6699 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.814s user 0.622s sys 1.135s
W20260810 07:54:28.084453 6724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:54:28.088033 6721 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1819 milliseconds
I20260810 07:54:28.088133 6699 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260810 07:54:28.089538 6699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:54:28.091979 6699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:54:28.093371 6699 hybrid_clock.cc:648] HybridClock initialized: now 1786348468093332 us; error 84 us; skew 500 ppm
I20260810 07:54:28.094650 6699 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 07:54:28.102813 6699 webserver.cc:533] Webserver started at http://127.31.139.126:37953/ using document root <none> and password file <none>
I20260810 07:54:28.103914 6699 fs_manager.cc:362] Metadata directory not provided
I20260810 07:54:28.104223 6699 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:54:28.113166 6699 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 07:54:28.119237 6759 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:54:28.120690 6699 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 07:54:28.121136 6699 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data,/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
uuid: "33049f5d320f4c6ca8d040c13fa09252"
format_stamp: "Formatted at 2026-08-10 07:54:03 on dist-test-slave-x5fp"
I20260810 07:54:28.123714 6699 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:54:28.189003 6699 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260810 07:54:28.190693 6699 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:54:28.191191 6699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:54:28.305543 6699 rpc_server.cc:307] RPC server started. Bound to: 127.31.139.126:38803
I20260810 07:54:28.305626 6843 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.139.126:38803 every 8 connection(s)
I20260810 07:54:28.308838 6699 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/master-0/data/info.pb
I20260810 07:54:28.311849 32301 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskdk4jGL/build/tsan/bin/kudu as pid 6699
I20260810 07:54:28.329272 6845 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:54:28.338987 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Bootstrap starting.
I20260810 07:54:28.366521 6845 log.cc:826] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Log is configured to *not* fsync() on all Append() calls
I20260810 07:54:28.439839 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=16 ignored=0} mutations{seen=25 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:54:28.440855 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: Bootstrap complete.
I20260810 07:54:28.459685 6845 raft_consensus.cc:359] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:28.461912 6845 raft_consensus.cc:740] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33049f5d320f4c6ca8d040c13fa09252, State: Initialized, Role: FOLLOWER
I20260810 07:54:28.462842 6845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:28.465610 6854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.466408 6854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:28.466917 6845 sys_catalog.cc:565] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:54:28.504702 6874 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:28.506155 6874 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: loading cluster ID for follower catalog manager: success
I20260810 07:54:28.522549 6874 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: acquiring CA information for follower catalog manager: success
I20260810 07:54:28.535728 6874 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:54:28.568563 5905 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:28.574183 6753 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:28.574712 6753 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
W20260810 07:54:28.580849 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:28.581425 6880 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
W20260810 07:54:28.582177 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:28.582643 5489 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:28.585618 6753 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:28.586100 6753 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:28.589030 6880 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:28.604565 6880 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
I20260810 07:54:28.550904 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:28.615674 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:28.617053 6789 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" instance_seqno: 1786348455229866) as {username='slave'} at 127.31.139.66:35829; Asking this server to re-register.
I20260810 07:54:28.619370 5905 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:28.626422 6788 ts_manager.cc:194] Registered new tserver with Master: 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
I20260810 07:54:28.629338 6806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252" is_pre_election: true
I20260810 07:54:28.630657 6806 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a552a496f9e0418d97a2006bdef0af5a in term 2.
I20260810 07:54:28.632645 6429 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a; no voters:
I20260810 07:54:28.637179 6880 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260810 07:54:28.637535 6880 raft_consensus.cc:493] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Starting leader election (detected failure of leader c4cb37d55c894071b60fe8c11447e918)
I20260810 07:54:28.637877 6880 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 3 FOLLOWER]: Advancing to term 4
W20260810 07:54:28.642237 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:28.645550 6880 raft_consensus.cc:515] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:28.648358 6880 leader_election.cc:290] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 election: Requested vote from peers 33049f5d320f4c6ca8d040c13fa09252 (127.31.139.126:38803), c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987)
I20260810 07:54:28.650123 6806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a552a496f9e0418d97a2006bdef0af5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: false dest_uuid: "33049f5d320f4c6ca8d040c13fa09252"
I20260810 07:54:28.650606 6806 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 2 FOLLOWER]: Advancing to term 4
W20260810 07:54:28.654055 6427 leader_election.cc:336] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:28.658288 6806 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a552a496f9e0418d97a2006bdef0af5a in term 4.
I20260810 07:54:28.659387 6429 leader_election.cc:304] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252, a552a496f9e0418d97a2006bdef0af5a; no voters: c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:28.660203 6880 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 4 FOLLOWER]: Leader election won for term 4
I20260810 07:54:28.661513 6880 raft_consensus.cc:697] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [term 4 LEADER]: Becoming Leader. State: Replica: a552a496f9e0418d97a2006bdef0af5a, State: Running, Role: LEADER
I20260810 07:54:28.662487 6880 consensus_queue.cc:237] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:28.667233 6886 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552a496f9e0418d97a2006bdef0af5a. Latest consensus state: current_term: 4 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.667954 6886 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:28.670617 6889 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260810 07:54:28.676736 6889 catalog_manager.cc:679] Loaded metadata for table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac]
I20260810 07:54:28.678211 6889 catalog_manager.cc:679] Loaded metadata for table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e]
I20260810 07:54:28.679412 6889 catalog_manager.cc:679] Loaded metadata for table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd]
I20260810 07:54:28.685514 6889 tablet_loader.cc:96] loaded metadata for tablet b5ed3ebcc27c499aaaf8d33568b7d6f7 (table default.table_2baacc33b03b46a9a286be0f0e9abe90 [id=32f58349ec154de5a8215c8a3b0a409e])
I20260810 07:54:28.686589 6889 tablet_loader.cc:96] loaded metadata for tablet eaac9de003b44d7288eb4194b2326cd0 (table default.table_d72b29e6aad6422384022c20cf87f343 [id=2042666a41234cc3922f6ba5e13cd1ac])
I20260810 07:54:28.687532 6889 tablet_loader.cc:96] loaded metadata for tablet fac4b0506a164a46b66a7328bef23d91 (table default.table_fa2edebca3114f2b90a6b3f6a79f1699 [id=ef3856c6f3044998ac5d4545c02ea0fd])
I20260810 07:54:28.688452 6889 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260810 07:54:28.692168 6889 catalog_manager.cc:1295] Loaded cluster ID: e3b9e9b6de7a4a5e9193dc761489dc6a
I20260810 07:54:28.692350 6889 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260810 07:54:28.696012 6889 catalog_manager.cc:1540] Loading token signing keys...
I20260810 07:54:28.699154 6889 catalog_manager.cc:6103] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Loaded TSK: 0
I20260810 07:54:28.700176 6889 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260810 07:54:28.703212 6889 catalog_manager.cc:1563] Loading latest processed Hive Metastore notification log event ID...
I20260810 07:54:28.706346 6889 catalog_manager.cc:5941] Last processed Hive Metastore notification event ID: -1
I20260810 07:54:28.721391 6806 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Refusing update from remote peer a552a496f9e0418d97a2006bdef0af5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20260810 07:54:28.724437 6886 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 28, Time since last communication: 0.000s
W20260810 07:54:28.726284 6427 consensus_peers.cc:597] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a -> Peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Couldn't send request to peer c4cb37d55c894071b60fe8c11447e918. Status: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:54:28.750644 5673 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:28.757231 6140 heartbeater.cc:344] Connected to a master server at 127.31.139.126:38803
I20260810 07:54:28.757854 6454 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59940:
table { table_name: "default.Table_d72b29e6aad6422384022c20cf87f343" } new_table_name: "default.Table_58cc5e9b175943a39b188b4040251b54" modify_external_catalogs: true
I20260810 07:54:28.767102 6453 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59940:
table { table_name: "default.Table_fa2edebca3114f2b90a6b3f6a79f1699" } new_table_name: "default.Table_2421dcedd74043649cacf204a2130f13" modify_external_catalogs: true
I20260810 07:54:28.772888 6450 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59940:
name: "default.Table_eef964168fae4ea3865d27a32a3af580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260810 07:54:28.771039 6539 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_ea1051c6abd749979cb4e6c577078521, message={"server":"","servicePrincipal":"","db":"default","table":"table_ea1051c6abd749979cb4e6c577078521","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_ea1051c6abd749979cb4e6c577078521\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348457},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_ea1051c6abd749979cb4e6c577078521\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_ea1051c6abd749979cb4e6c577078521\",\"kudu.table_id\":\"464515f666b04ceaa8ffab909babac70\",\"transient_lastDdlTime\":\"1786348457\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1786348458, eventType=CREATE_TABLE, dbName=default, tableName=table_d72b29e6aad6422384022c20cf87f343, message={"server":"","servicePrincipal":"","db":"default","table":"table_d72b29e6aad6422384022c20cf87f343","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_d72b29e6aad6422384022c20cf87f343\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1786348458},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskdk4jGL/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786348349737359-32301-0/minicluster-data/warehouse/table_d72b29e6aad6422384022c20cf87f343\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_d72b29e6aad6422384022c20cf87f343\",\"kudu.table_id\":\"2042666a41234cc3922f6ba5e13cd1ac\",\"transient_lastDdlTime\":\"1786348458\",\"comment\":\"\",\"kudu.master_addresses\":\"127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e3b9e9b6de7a4a5e9193dc761489dc6a\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1786348458,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260810 07:54:28.772643 6451 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59940:
name: "default.Table_c1ffabac25f54e5eb1d0225b2ea9ac75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:28.775452 6440 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59940:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
I20260810 07:54:28.778777 6446 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59940:
name: "default.Table_17d9a8f6ddca4ecd861be1837d81ac39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:28.780550 6445 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f03ff5177244b3da7165d3200b945ea from {username='slave'} at 127.0.0.1:59940
I20260810 07:54:28.780691 6447 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f03ff5177244b3da7165d3200b945ea from {username='slave'} at 127.0.0.1:59940
I20260810 07:54:28.788691 6898 mvcc.cc:204] Tried to move back new op lower bound from 7316883305625243648 to 7316883304223346688. Current Snapshot: MvccSnapshot[applied={T|T < 7316883305625243648 or (T in {7316883305625243648})}]
I20260810 07:54:28.789682 6441 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59940:
name: "default.Table_25f242ffd8e64a9b956f1b4a88b60eb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:54:28.794777 6440 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59940:
table { table_name: "default.Table_2baacc33b03b46a9a286be0f0e9abe90" } modify_external_catalogs: true
I20260810 07:54:28.801308 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.810899 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:28.809823 6787 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" instance_seqno: 1786348457427846) as {username='slave'} at 127.31.139.67:43813; Asking this server to re-register.
I20260810 07:54:28.802631 6854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552a496f9e0418d97a2006bdef0af5a. Latest consensus state: current_term: 4 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.814280 6854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:28.812107 6789 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" instance_seqno: 1786348452360200) as {username='slave'} at 127.31.139.65:49073; Asking this server to re-register.
I20260810 07:54:28.815402 6140 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:28.815310 6886 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.816349 6886 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:54:28.816495 5673 heartbeater.cc:461] Registering TS with master...
I20260810 07:54:28.824626 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "a552a496f9e0418d97a2006bdef0af5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } } }
I20260810 07:54:28.825356 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:54:28.830544 6786 ts_manager.cc:194] Registered new tserver with Master: d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779)
I20260810 07:54:28.831238 6787 ts_manager.cc:194] Registered new tserver with Master: 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
@ 0x7ff1fc099920 google::LogMessage::Flush() at ??:0
@ 0x7ff1fc09968f google::LogMessage::~LogMessage() at ??:0
@ 0x7ff207219fbc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7ff20721977c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7ff20721d7f2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7ff20721d7aa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
@ 0x7ff20721d73a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7ff20721d702 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7ff20721c9fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd8c5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd6fa std::__1::function<>::operator()() at ??:?
I20260810 07:54:28.995828 6445 master_service.cc:960] ReplaceTablet: received request to replace tablet 21b743816cbb4754925a5e1b767f0ce7 from {username='slave'} at 127.0.0.1:59940
@ 0x7ff1fe2831eb kudu::Thread::SuperviseThread() at ??:0
@ 0x4568e0 __tsan_thread_start_func at ??:?
I20260810 07:54:29.005336 6445 master_service.cc:960] ReplaceTablet: received request to replace tablet eaac9de003b44d7288eb4194b2326cd0 from {username='slave'} at 127.0.0.1:59940
@ 0x7ff1ff9e36db start_thread at ??:0
I20260810 07:54:29.027547 6445 catalog_manager.cc:6006] T 00000000000000000000000000000000 P a552a496f9e0418d97a2006bdef0af5a: Sending DeleteTablet for 3 replicas of tablet eaac9de003b44d7288eb4194b2326cd0
@ 0x7ff1fad3e61f clone at ??:0
I20260810 07:54:29.067993 6139 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
W20260810 07:54:29.251812 5716 connection.cc:573] client connection to 127.31.139.124:41801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:29.253304 6928 negotiation.cc:336] Failed RPC negotiation. Trace:
0810 07:54:29.031396 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.31.139.66:38495)
0810 07:54:29.034909 (+ 3513us) server_negotiation.cc:207] Beginning negotiation
0810 07:54:29.034949 (+ 40us) server_negotiation.cc:400] Waiting for connection header
0810 07:54:29.035007 (+ 58us) server_negotiation.cc:408] Connection header received
0810 07:54:29.035177 (+ 170us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0810 07:54:29.035210 (+ 33us) server_negotiation.cc:462] Received NEGOTIATE request from client
0810 07:54:29.035507 (+ 297us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0810 07:54:29.035687 (+ 180us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0810 07:54:29.251914 (+216227us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51294: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":2736,"thread_start_us":2451,"threads_started":1}
W20260810 07:54:29.253404 6154 connection.cc:573] client connection to 127.31.139.124:41801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:29.255326 6154 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.31.139.124:41801)
W20260810 07:54:29.255357 5955 connection.cc:573] client connection to 127.31.139.124:41801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:54:29.256040 6154 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.31.139.124:41801)
W20260810 07:54:29.256682 6154 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.139.124:41801)
W20260810 07:54:29.258545 6154 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.31.139.124:41801)
I20260810 07:54:29.258980 5668 heartbeater.cc:499] Master 127.31.139.124:41801 was elected leader, sending a full tablet report...
I20260810 07:54:29.534179 6937 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:29.534649 6937 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:29.537549 6937 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:29.544472 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:29.550326 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:29.551083 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:29.551461 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:29.552135 6937 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:29.750341 6948 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:29.751085 6948 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:29.753280 6948 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:54:29.755652 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:29.756418 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:29.756937 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:29.758764 6948 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:30.266944 6954 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:30.267530 6954 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:30.269814 6954 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:30.277184 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:30.277755 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:30.278276 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:30.279040 6954 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:30.709709 32301 master-stress-test.cc:579] Killing test thread 1/10
I20260810 07:54:30.813840 6964 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:30.814301 6964 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:30.815876 6964 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:30.821506 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:30.822033 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:30.822408 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:30.823012 6964 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:54:31.223881 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 6: this message will repeat every 5th retry.
I20260810 07:54:32.356623 6983 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:32.357223 6983 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:32.359395 6983 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:32.365710 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:32.366338 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:32.366756 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:32.367527 6983 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:33.621464 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:33.621948 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:33.856647 7013 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:33.857197 7013 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:33.858873 7013 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:54:33.861006 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:33.861469 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:33.861822 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:33.862592 7015 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:33.905265 7016 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:33.905679 7016 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:33.907210 7016 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:33.911540 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:33.911959 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:33.912312 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:33.912997 7016 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:34.442646 7037 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:34.443089 7037 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:34.445029 7037 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:34.450942 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:34.451946 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:34.452289 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:34.452986 7037 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:34.813833 7013 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:34.814337 7013 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:34.816287 7013 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:54:34.818056 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:34.818572 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:34.818955 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:34.819598 7013 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:35.582000 7052 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:35.582422 7052 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:35.584080 7052 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:35.586426 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260810 07:54:35.589722 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:35.590948 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:35.591243 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:35.591794 7052 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:36.324936 7079 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:36.325820 7079 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:36.328390 7079 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:36.337860 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:54:36.338598 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:54:36.339037 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:36.339887 7079 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:38.627722 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:38.628301 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:54:41.989230 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 11: this message will repeat every 5th retry.
I20260810 07:54:42.407909 7155 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:42.408617 7155 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:42.411157 7155 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:54:42.412420 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:42.413087 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:42.413502 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:42.414242 7155 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:43.634027 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:43.634418 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:43.814365 7155 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:43.814844 7155 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:43.816430 7155 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:54:43.818368 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:43.818847 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:43.819229 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:43.819948 7155 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:54:47.889652 7282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:47.890291 7282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:47.891814 7282 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:47.895046 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:54:47.899194 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:47.899747 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:47.900135 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:47.900867 7282 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:48.640375 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:48.640872 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:51.269366 7350 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:51.270066 7350 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:54:51.272459 7350 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:54:51.274560 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:54:51.275085 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:54:51.275430 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:51.276091 7350 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:54:52.491199 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 16: this message will repeat every 5th retry.
I20260810 07:54:53.290055 7389 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:54:53.290493 7389 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:54:53.292171 7389 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:54:53.294502 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:54:53.297814 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:54:53.298292 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:54:53.298681 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:54:53.299238 7389 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:54:53.645874 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:53.646235 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:54:54.030054 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf120 after lost signal to thread 5683
W20260810 07:54:54.031003 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000de180 after lost signal to thread 5901
I20260810 07:54:58.652767 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:54:58.653333 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:54:59.370544 7498 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:54:59.371022 7498 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:54:59.372519 7498 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:54:59.374240 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:54:59.374882 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:54:59.375283 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:54:59.376016 7498 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:55:02.813958 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 21: this message will repeat every 5th retry.
I20260810 07:55:03.660041 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:03.660496 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:07.089695 7595 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:07.090353 7595 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:55:07.092679 7595 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:55:07.095584 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:55:07.096252 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:55:07.096547 5489 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:07.097225 7595 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:55:07.219856 7595 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:07.220395 7595 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:55:07.222071 7595 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:55:07.223932 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:55:07.224416 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:55:07.224843 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:07.225471 7595 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:55:08.668424 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:08.669014 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:08.883731 7661 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:08.884333 7661 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:08.886654 7661 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:08.888162 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:55:08.891460 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:08.892438 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:08.892827 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:08.893533 7661 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:55:10.531761 7694 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:10.532169 7694 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:10.533996 7694 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:10.538537 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:10.539310 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:10.539660 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:10.540270 7694 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:55:11.413859 7718 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:11.414582 7718 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:55:11.417042 7718 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:55:11.418371 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:55:11.419006 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:55:11.419330 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:11.419950 7718 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:55:12.229579 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 26: this message will repeat every 5th retry.
I20260810 07:55:13.674590 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:13.675007 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:55:18.002177 5904 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (97 consecutive failures): Network error: Failed to ping master at 127.31.139.124:41801: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:55:18.150074 5668 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (97 consecutive failures): Network error: Failed to ping master at 127.31.139.124:41801: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:55:18.229332 6139 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (121 consecutive failures): Network error: Failed to ping master at 127.31.139.124:41801: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:18.681535 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:18.682054 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:55:21.703627 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 31: this message will repeat every 5th retry.
I20260810 07:55:23.688604 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:23.689116 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:25.737721 7944 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:25.738456 7944 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:55:25.740607 7944 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:55:25.742448 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:55:25.743093 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:55:25.743453 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:25.744345 7944 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:55:26.839989 5660 debug-util.cc:398] Leaking SignalData structure 0x7b080006d400 after lost signal to thread 5412
W20260810 07:55:26.841078 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7f20 after lost signal to thread 5665
I20260810 07:55:27.223819 7984 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:27.224438 7984 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:27.226939 7984 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:27.229651 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:55:27.233290 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:27.234073 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:27.234409 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:27.235026 7984 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:55:27.797729 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8c20 after lost signal to thread 6701
W20260810 07:55:27.798609 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7780 after lost signal to thread 6843
I20260810 07:55:28.734272 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:28.734923 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:55:29.834826 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd020 after lost signal to thread 5412
W20260810 07:55:29.836539 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000de3c0 after lost signal to thread 5665
I20260810 07:55:30.653872 7971 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:30.654439 7971 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:55:30.656213 7971 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:55:30.657905 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:55:30.658460 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:55:30.658717 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:30.659236 7971 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:55:31.668787 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 36: this message will repeat every 5th retry.
I20260810 07:55:33.741168 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:33.741593 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:55:34.098570 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000da960 after lost signal to thread 5920
W20260810 07:55:34.099618 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0aa0 after lost signal to thread 6129
I20260810 07:55:35.845762 8122 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:35.846331 8122 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:55:35.848629 8122 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:55:35.850474 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:55:35.850931 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:55:35.851227 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:35.851821 8122 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:55:38.747926 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:38.748461 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:39.637467 8178 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:39.638130 8178 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:55:39.640125 8178 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:55:39.641650 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:55:39.642311 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:55:39.642637 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:39.643316 8178 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:55:42.274402 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 41: this message will repeat every 5th retry.
I20260810 07:55:43.754329 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:43.754799 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:43.767988 8272 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:43.768764 8272 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:43.771260 8272 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:43.772763 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:55:43.776280 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:43.776906 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:43.777199 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:43.777818 8272 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:55:46.617012 8264 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:46.617475 8264 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:55:46.619614 8264 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:55:46.621369 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:55:46.621920 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:55:46.622252 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:46.622835 8264 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:55:48.022608 8348 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:48.023264 8348 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:48.025477 8348 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:48.031721 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:48.032845 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:48.033210 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:48.033846 8348 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:55:48.760632 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:48.761174 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:51.993417 8414 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:51.993908 8414 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:51.995556 8414 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:51.997956 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:55:52.001345 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:55:52.001823 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:55:52.002192 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:52.002791 8414 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:55:52.201768 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 46: this message will repeat every 5th retry.
I20260810 07:55:53.041127 8264 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:55:53.041935 8264 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:55:53.044176 8264 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:55:53.045975 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:55:53.046548 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:55:53.046854 5493 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:55:53.047457 8264 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:55:53.766731 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:53.767120 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:55:58.698395 8498 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:55:58.699738 8498 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:55:58.712549 8498 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:55:58.717427 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:55:58.724597 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:55:58.725517 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:55:58.726065 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:55:58.727645 8498 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:55:58.778641 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:55:58.779512 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:00.145289 8481 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:00.145880 8481 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:56:00.147863 8481 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:56:00.149645 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:56:00.150199 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:56:00.150517 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:00.151113 8481 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:56:00.501910 8481 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:00.502557 8481 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:56:00.504369 8481 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:56:00.505632 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:56:00.506822 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:56:00.507174 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:00.507929 8481 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:56:02.150522 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 51: this message will repeat every 5th retry.
W20260810 07:56:03.727653 5898 debug-util.cc:398] Leaking SignalData structure 0x7b080006c720 after lost signal to thread 5683
W20260810 07:56:03.729415 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000dca00 after lost signal to thread 5901
I20260810 07:56:03.785761 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:03.786193 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:56:07.908527 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000403a0 after lost signal to thread 5920
W20260810 07:56:07.909937 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3840 after lost signal to thread 6129
W20260810 07:56:08.391031 5660 debug-util.cc:398] Leaking SignalData structure 0x7b0800083a20 after lost signal to thread 5412
W20260810 07:56:08.391949 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000df9e0 after lost signal to thread 5665
I20260810 07:56:08.793486 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:08.794078 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:12.628422 8641 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:12.629050 8641 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:56:12.631386 8641 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:56:12.632910 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:56:12.633525 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:56:12.633775 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:12.634342 8641 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:56:12.662104 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 56: this message will repeat every 5th retry.
I20260810 07:56:13.800448 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:13.800915 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:16.454627 8712 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:56:16.455401 8712 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:56:16.458361 8712 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:56:16.462208 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:56:16.466167 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:56:16.466707 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:56:16.467195 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:56:16.467840 8712 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:56:17.024320 8674 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:17.024845 8674 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:56:17.026458 8674 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:56:17.028050 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:56:17.028476 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:56:17.028810 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:17.029398 8674 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:56:17.668743 6840 debug-util.cc:398] Leaking SignalData structure 0x7b0800070120 after lost signal to thread 6701
W20260810 07:56:17.669329 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8e20 after lost signal to thread 6843
W20260810 07:56:18.086438 5902 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (230 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:56:18.593415 6139 heartbeater.cc:646] Failed to heartbeat to 127.31.139.124:41801 (269 consecutive failures): Network error: Failed to ping master at 127.31.139.124:41801: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:56:18.598191 5666 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (231 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:56:18.862234 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:18.862808 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:19.778676 8777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:56:19.779158 8777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:56:19.781268 8777 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:56:19.787318 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:56:19.787822 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:56:19.788241 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:56:19.789036 8777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:56:22.302003 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 61: this message will repeat every 5th retry.
I20260810 07:56:23.868438 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:23.868976 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:25.369324 8785 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:25.370021 8785 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:56:25.371882 8785 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:56:25.373637 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:56:25.374166 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:56:25.374464 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:25.375022 8785 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:56:26.898381 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6240 after lost signal to thread 5683
W20260810 07:56:26.899399 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000adec0 after lost signal to thread 5901
W20260810 07:56:27.950240 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d62e0 after lost signal to thread 5683
W20260810 07:56:27.951084 5898 debug-util.cc:398] Leaking SignalData structure 0x7b0800040a40 after lost signal to thread 5901
I20260810 07:56:28.532392 6770 maintenance_manager.cc:419] P 33049f5d320f4c6ca8d040c13fa09252: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033352
I20260810 07:56:28.615000 6769 maintenance_manager.cc:643] P 33049f5d320f4c6ca8d040c13fa09252: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.081s user 0.047s sys 0.009s Metrics: {"bytes_written":11180,"cfile_init":1,"drs_written":1,"lbm_read_time_us":218,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2248,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":19,"thread_start_us":478,"threads_started":1,"wal-append.queue_time_us":832}
I20260810 07:56:28.617552 6770 maintenance_manager.cc:419] P 33049f5d320f4c6ca8d040c13fa09252: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1134 bytes on disk
I20260810 07:56:28.619472 6769 maintenance_manager.cc:643] P 33049f5d320f4c6ca8d040c13fa09252: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":257,"lbm_reads_lt_1ms":4}
I20260810 07:56:28.875638 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:28.876078 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:56:31.509262 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9cc0 after lost signal to thread 5683
W20260810 07:56:31.510056 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d65a0 after lost signal to thread 5901
W20260810 07:56:32.050014 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 66: this message will repeat every 5th retry.
I20260810 07:56:32.762219 8997 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:56:32.763027 8997 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:56:32.765832 8997 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:56:32.769258 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:56:32.773097 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:56:32.773746 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:56:32.774178 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:56:32.774816 8997 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:56:33.882099 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:33.882516 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:35.718333 9023 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:35.719234 9023 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:56:35.721560 9023 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:56:35.723340 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:56:35.723887 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:56:35.724255 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:35.724946 9023 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:56:38.888564 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:38.889137 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:56:41.664249 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0ac0 after lost signal to thread 5412
W20260810 07:56:41.665266 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5a40 after lost signal to thread 5665
W20260810 07:56:42.146507 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 71: this message will repeat every 5th retry.
I20260810 07:56:42.740278 9203 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:42.740983 9203 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:56:42.744374 9203 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:56:42.745028 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:56:42.745671 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:56:42.745980 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:42.746649 9203 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:56:43.894893 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:43.895335 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:48.902298 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:48.902890 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:51.951380 9437 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:56:51.951942 9437 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:56:51.954039 9437 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:56:51.956897 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:56:51.961133 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:56:51.961745 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:56:51.962100 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:56:51.962877 9437 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:56:52.942903 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 76: this message will repeat every 5th retry.
I20260810 07:56:53.154451 9391 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:56:53.155354 9391 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:56:53.158812 9391 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:56:53.160300 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:56:53.167461 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:56:53.167840 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:56:53.168574 9391 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:56:53.908682 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:53.909103 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:56:58.915154 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:56:58.915606 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:00.961086 9607 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:00.961647 9607 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:00.963596 9607 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:00.965205 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:00.965832 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:00.966116 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:00.966706 9607 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:02.233358 9607 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:02.233916 9607 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:02.235860 9607 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:02.237427 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:02.237928 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:02.238350 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:02.238993 9607 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:03.178200 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 81: this message will repeat every 5th retry.
I20260810 07:57:03.921267 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:03.921618 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:04.965090 9607 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:04.965489 9607 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:04.967314 9607 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:04.969116 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:04.969547 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:04.969882 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:04.970434 9607 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:07.925945 5898 debug-util.cc:398] Leaking SignalData structure 0x7b0800040ea0 after lost signal to thread 5683
W20260810 07:57:07.927166 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9580 after lost signal to thread 5901
I20260810 07:57:08.333493 9791 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:08.334196 9791 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:08.336521 9791 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:08.339375 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:57:08.342803 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:08.343427 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:57:08.343776 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:08.344493 9791 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:57:08.928088 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:08.928602 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:10.105356 9778 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:10.106096 9778 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:10.108891 9778 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:10.110750 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:10.111462 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:10.111734 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:10.112391 9778 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:12.664198 9875 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:12.664819 9875 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:12.666749 9875 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:12.668526 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:12.669003 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:12.669412 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:12.670079 9875 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:13.840509 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 86: this message will repeat every 5th retry.
I20260810 07:57:13.934227 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:13.934666 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:57:18.368357 5902 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (350 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:57:18.943163 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:18.943806 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:57:18.984378 6138 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (439 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:19.046267 5666 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (351 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:57:21.156525 10095 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:21.157155 10095 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:21.158887 10095 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:21.161697 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:57:21.165807 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:57:21.166388 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:57:21.166843 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:21.167627 10095 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:57:22.896436 10114 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:22.897199 10114 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:22.899210 10114 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:22.900974 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:22.901515 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:22.901829 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:22.902433 10114 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:23.464838 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 91: this message will repeat every 5th retry.
I20260810 07:57:23.950565 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:23.951021 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:57:24.661024 5660 debug-util.cc:398] Leaking SignalData structure 0x7b0800040b20 after lost signal to thread 5412
W20260810 07:57:24.661931 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3ac0 after lost signal to thread 5665
I20260810 07:57:28.958909 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:28.959489 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:32.201237 10299 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:32.202008 10299 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:32.204248 10299 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:32.206095 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:32.206607 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:32.206948 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:32.207571 10299 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:32.946424 10351 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:32.947038 10351 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:32.955446 10351 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:32.960318 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:57:32.964468 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:32.965399 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:57:32.965778 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:32.966531 10351 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:57:33.534210 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 96: this message will repeat every 5th retry.
I20260810 07:57:33.969164 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:33.969995 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:38.976925 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:38.977494 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:39.758148 10503 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:39.758684 10503 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:39.760509 10503 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:39.762463 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:39.762899 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:39.763208 5489 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:39.763926 10503 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:41.073400 10537 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:41.074209 10537 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:41.076120 10537 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:41.077890 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:41.078516 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:41.078836 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:41.079443 10537 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:41.108919 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2d00 after lost signal to thread 5920
W20260810 07:57:41.109920 6126 debug-util.cc:398] Leaking SignalData structure 0x7b080003e420 after lost signal to thread 6129
W20260810 07:57:42.158576 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2520 after lost signal to thread 5920
W20260810 07:57:42.159602 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5880 after lost signal to thread 6129
I20260810 07:57:43.932078 10599 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:43.932585 10599 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:43.934698 10599 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:43.935884 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:43.936285 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:43.936609 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:43.937206 10599 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:43.968392 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 101: this message will repeat every 5th retry.
I20260810 07:57:43.983186 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:43.983635 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:46.185580 10652 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:46.186131 10652 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:46.189153 10652 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:46.191380 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:57:46.195227 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:46.195822 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:57:46.196158 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:46.196842 10652 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:57:48.520530 10701 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:48.521304 10701 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:48.523257 10701 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:48.529646 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:48.530265 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:57:48.530598 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:48.531208 10701 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:57:48.989025 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:48.989490 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:50.379741 10745 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:50.380528 10745 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:57:50.383513 10745 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:57:50.385563 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:57:50.386842 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:57:50.387311 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:50.388482 10745 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:50.767308 10745 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:50.767844 10745 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:50.769821 10745 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:50.771521 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:50.771948 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:50.772300 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:50.772933 10745 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:57:53.312806 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 106: this message will repeat every 5th retry.
I20260810 07:57:53.995039 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:53.995425 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:57:55.395654 10767 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:57:55.396176 10767 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:57:55.397826 10767 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:57:55.399641 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:57:55.400048 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:57:55.400365 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:57:55.400969 10767 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:57:58.278370 10876 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:57:58.278929 10876 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:57:58.280936 10876 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:57:58.283571 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:57:58.286800 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:57:58.287240 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:57:58.287556 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:57:58.288206 10876 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:57:59.001739 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:57:59.002265 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:02.724684 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 111: this message will repeat every 5th retry.
I20260810 07:58:04.007880 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:04.008242 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:58:06.249588 11020 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:06.250347 11020 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:58:06.253477 11020 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:58:06.254933 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:58:06.256083 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:58:06.256377 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:06.257323 11020 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:06.749191 11020 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:06.749737 11020 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:06.751868 11020 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:06.753543 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:06.754029 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:06.754418 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:06.755085 11020 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:09.015012 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:09.015447 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:58:11.782115 11143 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:11.782814 11143 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:11.785005 11143 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:11.786649 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:11.787192 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:11.787451 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:11.788029 11143 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:12.325112 11170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:12.325706 11170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:12.327606 11170 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:12.329759 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:58:12.333437 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:58:12.333911 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:58:12.334232 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:12.334785 11170 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:58:12.378782 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000707c0 after lost signal to thread 6701
W20260810 07:58:12.379426 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8920 after lost signal to thread 6843
W20260810 07:58:13.299261 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 116: this message will repeat every 5th retry.
I20260810 07:58:14.152530 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:14.153028 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:18.664983 5902 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (470 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:18.992476 6138 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (589 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:58:19.159519 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:19.159996 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:19.511566 5666 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (471 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:58:23.612075 11396 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:23.613010 11396 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:23.615772 11396 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:23.617693 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:23.618470 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:23.618821 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:23.619562 11396 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:24.088297 11408 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:24.088970 11408 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:24.090696 11408 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:24.092931 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:58:24.096371 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:24.096903 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:58:24.097337 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:24.098079 11408 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:58:24.165387 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:24.165774 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:24.632244 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 121: this message will repeat every 5th retry.
I20260810 07:58:26.011149 11420 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:26.011649 11420 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:58:26.013522 11420 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:58:26.015154 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:58:26.015692 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:58:26.016034 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:26.016628 11420 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:27.242429 11420 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:27.242962 11420 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:27.244906 11420 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:27.247221 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:27.251053 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:27.251431 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:27.252065 11420 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:29.174368 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:29.174844 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:58:30.781589 11549 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:30.782370 11549 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:30.784832 11549 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:30.789139 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:58:30.792297 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:58:30.800313 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:58:30.800642 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:30.801323 11549 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:58:31.786952 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000e16e0 after lost signal to thread 5412
W20260810 07:58:31.787912 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000d0340 after lost signal to thread 5665
I20260810 07:58:34.181052 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:34.181474 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:34.906792 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 126: this message will repeat every 5th retry.
I20260810 07:58:35.691474 11642 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:35.691941 11642 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:35.693744 11642 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:35.699123 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:35.699867 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:58:35.700196 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:35.700853 11642 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:58:36.057338 11631 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:36.057992 11631 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:58:36.060161 11631 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:58:36.062033 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:58:36.062569 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:58:36.062887 5493 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:36.063640 11631 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:58:38.171892 6840 debug-util.cc:398] Leaking SignalData structure 0x7b0800070800 after lost signal to thread 6701
W20260810 07:58:38.172578 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000aea40 after lost signal to thread 6843
I20260810 07:58:38.834846 11708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:38.835493 11708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:38.839115 11708 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:38.841902 6763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:58:38.845307 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:38.845830 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:58:38.846199 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:38.846864 11708 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:58:39.203999 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7a60 after lost signal to thread 6701
W20260810 07:58:39.204487 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8d40 after lost signal to thread 6843
I20260810 07:58:39.246652 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:39.247231 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:58:43.209683 11773 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:43.210302 11773 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:58:43.212606 11773 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:58:43.214242 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:58:43.214885 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:58:43.215157 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:43.215764 11773 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:44.253818 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:44.254565 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:58:45.629549 11833 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:45.630013 11833 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:45.631901 11833 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:45.633458 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:45.633881 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:45.634204 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:45.634814 11833 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:58:45.824314 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 131: this message will repeat every 5th retry.
I20260810 07:58:47.670100 11876 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:47.670612 11876 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:58:47.672425 11876 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:58:47.674005 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:58:47.674453 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:58:47.674772 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:47.675314 11876 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:49.283912 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:49.284647 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:50.210776 6840 debug-util.cc:398] Leaking SignalData structure 0x7b0800070580 after lost signal to thread 6701
W20260810 07:58:50.211410 6840 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7ce0 after lost signal to thread 6843
I20260810 07:58:54.290548 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:54.291036 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:58:56.070956 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 136: this message will repeat every 5th retry.
I20260810 07:58:56.294801 12035 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:56.295444 12035 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:56.297758 12035 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:56.299865 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:58:56.303443 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:56.303972 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:58:56.304296 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:56.304936 12035 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:58:56.935214 12048 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:58:56.935932 12048 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:58:56.938097 12048 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:58:56.939961 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:58:56.940575 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:58:56.940872 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:58:56.941480 12048 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:58:57.448791 12060 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:58:57.449661 12060 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:58:57.452395 12060 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:58:57.460091 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:58:57.462098 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
I20260810 07:58:57.462437 6763 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:58:57.463259 12060 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260810 07:58:59.297756 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:58:59.298252 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:59:00.152912 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad400 after lost signal to thread 5683
W20260810 07:59:00.153882 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6280 after lost signal to thread 5901
I20260810 07:59:04.304112 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:04.304553 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:59:04.628006 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5740 after lost signal to thread 5920
W20260810 07:59:04.629104 6126 debug-util.cc:398] Leaking SignalData structure 0x7b08000c4e60 after lost signal to thread 6129
W20260810 07:59:05.987654 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 141: this message will repeat every 5th retry.
I20260810 07:59:07.607089 12275 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:59:07.607717 12275 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:59:07.610072 12275 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:59:07.611835 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:59:07.612459 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:59:07.612779 5489 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:59:07.613430 12275 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:59:09.311275 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:09.311829 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:59:13.484532 12405 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:59:13.485260 12405 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:59:13.487525 12405 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:59:13.489821 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260810 07:59:13.493109 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:59:13.493613 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:59:13.493938 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:59:13.494596 12405 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:59:13.774259 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad260 after lost signal to thread 5683
W20260810 07:59:13.775723 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d67c0 after lost signal to thread 5901
I20260810 07:59:14.317588 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:14.317967 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:59:15.225976 12393 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:59:15.226830 12393 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:59:15.229013 12393 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:59:15.230863 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:59:15.231492 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:59:15.231871 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:59:15.232579 12393 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:59:16.095561 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 146: this message will repeat every 5th retry.
W20260810 07:59:16.662602 5660 debug-util.cc:398] Leaking SignalData structure 0x7b08000419c0 after lost signal to thread 5412
W20260810 07:59:16.663528 5660 debug-util.cc:398] Leaking SignalData structure 0x7b0800040d80 after lost signal to thread 5665
W20260810 07:59:18.952920 5902 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (590 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:59:19.137051 6138 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (740 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:59:19.325018 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:19.325507 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:59:19.865864 12557 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a552a496f9e0418d97a2006bdef0af5a)
I20260810 07:59:19.866456 12557 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33049f5d320f4c6ca8d040c13fa09252" member_type: VOTER last_known_addr { host: "127.31.139.126" port: 38803 } } peers { permanent_uuid: "c4cb37d55c894071b60fe8c11447e918" member_type: VOTER last_known_addr { host: "127.31.139.125" port: 41987 } } peers { permanent_uuid: "a552a496f9e0418d97a2006bdef0af5a" member_type: VOTER last_known_addr { host: "127.31.139.124" port: 41801 } }
I20260810 07:59:19.868268 12557 leader_election.cc:290] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987), a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801)
W20260810 07:59:19.870721 6762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260810 07:59:19.873806 6763 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a552a496f9e0418d97a2006bdef0af5a (127.31.139.124:41801): Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:59:19.874279 6762 leader_election.cc:336] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c4cb37d55c894071b60fe8c11447e918 (127.31.139.125:41987): Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
I20260810 07:59:19.874629 6762 leader_election.cc:304] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33049f5d320f4c6ca8d040c13fa09252; no voters: a552a496f9e0418d97a2006bdef0af5a, c4cb37d55c894071b60fe8c11447e918
I20260810 07:59:19.875187 12557 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 33049f5d320f4c6ca8d040c13fa09252 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260810 07:59:19.945962 5666 heartbeater.cc:646] Failed to heartbeat to 127.31.139.125:41987 (591 consecutive failures): Network error: Failed to ping master at 127.31.139.125:41987: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111)
W20260810 07:59:22.019897 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3020 after lost signal to thread 5683
W20260810 07:59:22.020838 5898 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4780 after lost signal to thread 5901
I20260810 07:59:23.303092 12628 raft_consensus.cc:493] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:59:23.303891 12628 raft_consensus.cc:515] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } }
I20260810 07:59:23.306278 12628 leader_election.cc:290] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449), 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495)
W20260810 07:59:23.308079 5489 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
W20260810 07:59:23.308575 5493 leader_election.cc:343] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 1f03ff5177244b3da7165d3200b945ea
I20260810 07:59:23.308916 5493 leader_election.cc:304] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:59:23.309548 12628 raft_consensus.cc:2749] T 1f03ff5177244b3da7165d3200b945ea P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:59:24.331516 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:24.331965 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:59:25.404691 12628 raft_consensus.cc:493] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:59:25.405242 12628 raft_consensus.cc:515] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90b26857dd7400b8f9dbd9e3216dca3" member_type: VOTER last_known_addr { host: "127.31.139.65" port: 44779 } } peers { permanent_uuid: "7dcf6b9caf1d4099bbc8cf8b2c58f3b5" member_type: VOTER last_known_addr { host: "127.31.139.66" port: 38495 } } peers { permanent_uuid: "5c7100eee0ce4dae895ca9535696dfce" member_type: VOTER last_known_addr { host: "127.31.139.67" port: 37449 } }
I20260810 07:59:25.406936 12628 leader_election.cc:290] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495), 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449)
W20260810 07:59:25.408799 5489 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c7100eee0ce4dae895ca9535696dfce (127.31.139.67:37449): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
W20260810 07:59:25.409258 5493 leader_election.cc:343] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dcf6b9caf1d4099bbc8cf8b2c58f3b5 (127.31.139.66:38495): Not found: Tablet not found: 21b743816cbb4754925a5e1b767f0ce7
I20260810 07:59:25.409600 5493 leader_election.cc:304] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d90b26857dd7400b8f9dbd9e3216dca3; no voters: 5c7100eee0ce4dae895ca9535696dfce, 7dcf6b9caf1d4099bbc8cf8b2c58f3b5
I20260810 07:59:25.410158 12628 raft_consensus.cc:2749] T 21b743816cbb4754925a5e1b767f0ce7 P d90b26857dd7400b8f9dbd9e3216dca3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 07:59:25.693176 5951 consensus_peers.cc:597] T e57cbd8ce3af440ea7c907f91a66753b P 5c7100eee0ce4dae895ca9535696dfce -> Peer d90b26857dd7400b8f9dbd9e3216dca3 (127.31.139.65:44779): Couldn't send request to peer d90b26857dd7400b8f9dbd9e3216dca3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e57cbd8ce3af440ea7c907f91a66753b. This is attempt 151: this message will repeat every 5th retry.
I20260810 07:59:29.194260 6770 maintenance_manager.cc:419] P 33049f5d320f4c6ca8d040c13fa09252: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1134 bytes on disk
I20260810 07:59:29.206292 6769 maintenance_manager.cc:643] P 33049f5d320f4c6ca8d040c13fa09252: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.011s sys 0.000s Metrics: {"dirs.queue_time_us":815,"thread_start_us":914,"threads_started":2,"wal-append.queue_time_us":803}
I20260810 07:59:29.339128 6699 master_runner.cc:187] Error getting master registration for 127.31.139.125:41987: Network error: Client connection negotiation failed: client connection to 127.31.139.125:41987: connect: Connection refused (error 111),
I20260810 07:59:29.339694 6699 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:59:29.463526 6154 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:59:29.464046 6154 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:59:29.464321 6154 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
W20260810 07:59:29.464641 6154 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
F20260810 07:59:29.478408 6175 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786348769 (unix time) try "date -d @1786348769" if you are using GNU date ***
F20260810 07:59:29.479809 6176 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800007e2d) received by PID 32301 (TID 0x7f33657fe700) from PID 32301; stack trace: ***
@ 0x2b07a0 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b2c83 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f337b275980 (unknown) at ??:0
@ 0x7f337642fe87 gsignal at ??:0
@ 0x7f33764317f1 abort at ??:0
@ 0x2b6796 __interceptor_abort at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f33794c7d74 google::LogMessage::SendToLog() at ??:0
@ 0x7f33794c8920 google::LogMessage::Flush() at ??:0
@ 0x7f33794cda5b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x35f702 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x38c2fe _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846
@ 0x38c1b1 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:281
*** Check failure stack trace: ***
F20260810 07:59:29.479809 6176 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)F000F20260810 07:59:29.492544 6162 master-stress-test.cc:299] 000F20260810 07:59:29.495862 6163 master-stress-test.cc:299] 000F20260810 07:59:29.498875 6167 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)F @ 0x38b8fb std::__1::__thread_proxy<>()
*** Check failure stack trace: ***
@ 0x2b0660 __tsan_thread_start_func at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
F20260810 07:59:29.479809 6176 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)F000F20260810 07:59:29.492544 6162 master-stress-test.cc:299] 000F20260810 07:59:29.495862 6163 master-stress-test.cc:299] 000F20260810 07:59:29.498875 6167 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)FF20260810 07:59:29.486104 6168 master-stress-test.cc:299] F20260810 07:59:29.485986 6164 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c1ffabac25f54e5eb1d0225b2ea9ac75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.31.139.126:38803,127.31.139.125:41987,127.31.139.124:41801, num_attempts: 768) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.31.139.124:41801: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f337b26a6db start_thread at ??:0
@ 0x7f337651261f clone at ??:0