Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-09-26T02:00:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-26T02:00:52Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMwEE2d/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMwEE2d/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/security.properties
2026-09-26 02:00:59: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMwEE2d/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMwEE2d/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260926 02:01:07.545924 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--webserver_interface=127.29.89.190
--webserver_port=0
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:07.645965 30337 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:07.646158 30337 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:07.646196 30337 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:07.649493 30337 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:07.649546 30337 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:07.649566 30337 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:07.649585 30337 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:07.653934 30337 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.89.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:07.655035 30337 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:07.656122 30337 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:07.661432 30343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:07.661491 30345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:07.661592 30337 server_base.cc:1061] running on GCE node
W20260926 02:01:07.661684 30342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:07.662096 30337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:07.663028 30337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:07.664196 30337 hybrid_clock.cc:648] HybridClock initialized: now 1790388067664179 us; error 27 us; skew 500 ppm
I20260926 02:01:07.665701 30337 webserver.cc:533] Webserver started at http://127.29.89.190:41125/ using document root <none> and password file <none>
I20260926 02:01:07.666219 30337 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:07.666268 30337 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:07.666479 30337 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:07.667906 30337 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/instance:
uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.668308 30337 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal/instance:
uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.671355 30337 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 02:01:07.673391 30351 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:07.674549 30337 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:07.674669 30337 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.674762 30337 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:07.688828 30337 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:07.689394 30337 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:07.689580 30337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:07.696508 30337 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.190:45707
I20260926 02:01:07.696525 30403 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.190:45707 every 8 connection(s)
I20260926 02:01:07.697455 30337 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
I20260926 02:01:07.700306 30404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:07.702545 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30337
I20260926 02:01:07.702719 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal/instance
I20260926 02:01:07.703758 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:07.704093 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--webserver_interface=127.29.89.189
--webserver_port=0
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 02:01:07.720391 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
W20260926 02:01:07.721184 30353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:07.722095 30404 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:07.771348 30404 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } attempt: 1
W20260926 02:01:07.772898 30404 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:07.804613 30405 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:07.804807 30405 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:07.804844 30405 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:07.808259 30405 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:07.808311 30405 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:07.808331 30405 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:07.808349 30405 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:07.812150 30405 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.89.189
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:07.813128 30405 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:07.814131 30405 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:07.819569 30413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:07.819579 30416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:07.819877 30405 server_base.cc:1061] running on GCE node
W20260926 02:01:07.819595 30414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:07.820477 30405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:07.821336 30405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:07.822512 30405 hybrid_clock.cc:648] HybridClock initialized: now 1790388067822496 us; error 28 us; skew 500 ppm
I20260926 02:01:07.824298 30405 webserver.cc:533] Webserver started at http://127.29.89.189:44675/ using document root <none> and password file <none>
I20260926 02:01:07.824848 30405 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:07.824928 30405 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:07.825166 30405 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:07.826769 30405 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/instance:
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.827251 30405 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal/instance:
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.830288 30405 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 02:01:07.832609 30422 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:07.833472 30405 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 02:01:07.833601 30405 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.833698 30405 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:07.841071 30404 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } attempt: 2
W20260926 02:01:07.842451 30404 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:07.852238 30405 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:07.852768 30405 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:07.852900 30405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:07.860912 30474 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.189:33299 every 8 connection(s)
I20260926 02:01:07.860917 30405 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.189:33299
I20260926 02:01:07.861961 30405 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
I20260926 02:01:07.864612 30475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:07.867816 30475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:07.869624 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30405
I20260926 02:01:07.869766 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal/instance
I20260926 02:01:07.870553 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=0
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 02:01:07.876716 30475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:07.882947 30475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } has no permanent_uuid. Determining permanent_uuid...
W20260926 02:01:07.883665 30423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:07.884660 30475 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:07.933632 30404 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } attempt: 3
I20260926 02:01:07.933903 30475 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } attempt: 1
W20260926 02:01:07.935539 30475 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:07.938553 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } has no permanent_uuid. Determining permanent_uuid...
W20260926 02:01:07.939950 30404 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:07.971148 30404 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } attempt: 1
W20260926 02:01:07.972582 30404 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:07.973219 30477 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:07.973387 30477 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:07.973423 30477 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:07.976599 30477 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:07.976646 30477 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:07.976665 30477 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:07.976682 30477 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:07.980423 30477 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:07.981333 30477 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:07.982327 30477 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:07.987448 30484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:07.987461 30485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:07.987651 30487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:07.987543 30477 server_base.cc:1061] running on GCE node
I20260926 02:01:07.988166 30477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:07.989005 30477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:07.990172 30477 hybrid_clock.cc:648] HybridClock initialized: now 1790388067990168 us; error 34 us; skew 500 ppm
I20260926 02:01:07.991796 30477 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:07.992324 30477 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:07.992408 30477 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:07.992615 30477 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:07.994079 30477 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/instance:
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.994567 30477 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal/instance:
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:07.997592 30477 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260926 02:01:07.999686 30493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.000573 30477 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 02:01:08.000710 30477 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:08.000810 30477 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:08.003741 30475 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } attempt: 2
W20260926 02:01:08.005199 30475 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:08.012951 30477 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:08.013480 30477 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:08.013619 30477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:08.020882 30545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:08.020890 30477 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:08.021909 30477 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:08.024690 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:08.025949 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30477
I20260926 02:01:08.026058 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal/instance
I20260926 02:01:08.028726 30546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:08.037925 30546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:08.042898 30546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } has no permanent_uuid. Determining permanent_uuid...
I20260926 02:01:08.047783 30404 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } attempt: 2
I20260926 02:01:08.049410 30546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:08.052073 30546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:08.053056 30546 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:08.053562 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap starting.
I20260926 02:01:08.054600 30546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: No bootstrap required, opened a new log
I20260926 02:01:08.055675 30404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:08.056298 30404 log.cc:826] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:08.057196 30546 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.057420 30546 raft_consensus.cc:385] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:08.057518 30546 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:08.057744 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: No bootstrap required, opened a new log
I20260926 02:01:08.058045 30546 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.059099 30546 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:08.059388 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.059494 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.060477 30404 raft_consensus.cc:359] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.060636 30404 raft_consensus.cc:385] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:08.060724 30404 raft_consensus.cc:740] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 995d22fd6e1b4cdeb3b8aa4c61b56272, State: Initialized, Role: FOLLOWER
I20260926 02:01:08.061120 30404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.061569 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.061692 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.062273 30404 sys_catalog.cc:565] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 02:01:08.065244 30570 catalog_manager.cc:1607] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 02:01:08.065375 30570 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 02:01:08.063047 30477 master_runner.cc:187] Error getting master registration for 127.29.89.190:45707: OK, instance_id { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" instance_seqno: 1790388067694635 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 02:01:08.067655 30477 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:08.066972 30337 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: OK, instance_id { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" instance_seqno: 1790388067858585 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 02:01:08.069135 30337 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:01:08.072588 30585 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 02:01:08.072666 30585 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 02:01:08.096359 30475 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } attempt: 3
I20260926 02:01:08.109927 30475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap starting.
I20260926 02:01:08.112974 30475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:08.113899 30475 log.cc:826] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:08.119904 30475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: No bootstrap required, opened a new log
I20260926 02:01:08.123422 30475 raft_consensus.cc:359] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.123659 30475 raft_consensus.cc:385] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:08.123764 30475 raft_consensus.cc:740] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Initialized, Role: FOLLOWER
I20260926 02:01:08.124250 30475 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.124830 30590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.125021 30590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.126401 30475 sys_catalog.cc:565] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:08.136492 30405 master_runner.cc:264] No leader master found from master d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:08.136574 30405 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:01:08.147650 30604 catalog_manager.cc:1607] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 02:01:08.149850 30604 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 02:01:08.194464 30357 tablet.cc:2433] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 02:01:08.204056 30555 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:08.204218 30555 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.205309 30555 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:08.205935 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:08.206188 30378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 0.
I20260926 02:01:08.206812 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
I20260926 02:01:08.207047 30449 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 0.
I20260926 02:01:08.207693 30495 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:08.208478 30555 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:08.208545 30555 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:08.208586 30555 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:08.209949 30555 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.210294 30555 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 1 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:08.210994 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2"
I20260926 02:01:08.211107 30449 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:08.212176 30449 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 1.
I20260926 02:01:08.212550 30495 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:08.212790 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:08.212915 30378 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:08.213917 30378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 1.
I20260926 02:01:08.214409 30555 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:08.214700 30555 raft_consensus.cc:697] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 LEADER]: Becoming Leader. State: Replica: b285db155484443594c9385155358d4d, State: Running, Role: LEADER
I20260926 02:01:08.215068 30555 consensus_queue.cc:237] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.219188 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.219836 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:08.235450 30609 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:08.236517 30609 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:08.256350 30378 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Refusing update from remote peer b285db155484443594c9385155358d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 02:01:08.256350 30449 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Refusing update from remote peer b285db155484443594c9385155358d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 02:01:08.256971 30555 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:08.257411 30555 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:08.261108 30590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.261226 30590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.261690 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.261807 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 02:01:08.264686 30499 tablet.cc:2433] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 02:01:08.271767 30607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.271938 30607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:08.273567 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.273680 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.276091 30612 mvcc.cc:204] Tried to move back new op lower bound from 7333429527543406592 to 7333429527414730752. Current Snapshot: MvccSnapshot[applied={T|T < 7333429527543406592}]
I20260926 02:01:08.278903 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.279009 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:08.280177 30590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:08.280292 30590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:08.280897 30611 mvcc.cc:204] Tried to move back new op lower bound from 7333429527543406592 to 7333429527414730752. Current Snapshot: MvccSnapshot[applied={T|T < 7333429527543406592}]
I20260926 02:01:08.282490 30609 catalog_manager.cc:1396] Generated new cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:08.282542 30609 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:08.312664 30609 catalog_manager.cc:1419] Generated new certificate authority record
I20260926 02:01:08.313877 30609 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:08.332794 30609 catalog_manager.cc:6219] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Generated new TSK 0
I20260926 02:01:08.333566 30609 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:08.334252 30609 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:08.334789 30609 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 02:01:08.347378 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.89.129:0
--local_ip_for_outbound_sockets=127.29.89.129
--webserver_interface=127.29.89.129
--webserver_port=0
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260926 02:01:08.482820 30618 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:08.483038 30618 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:08.483089 30618 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:08.486400 30618 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:08.486500 30618 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.89.129
I20260926 02:01:08.490294 30618 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.89.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.89.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--never_fsync=true
--heap_profile_path=/tmp/kudu.30618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.89.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:08.491322 30618 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:08.492363 30618 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:08.498697 30626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:08.498690 30623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:08.498690 30624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:08.498798 30618 server_base.cc:1061] running on GCE node
I20260926 02:01:08.499320 30618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:08.499853 30618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:08.501055 30618 hybrid_clock.cc:648] HybridClock initialized: now 1790388068501040 us; error 31 us; skew 500 ppm
I20260926 02:01:08.502811 30618 webserver.cc:533] Webserver started at http://127.29.89.129:39447/ using document root <none> and password file <none>
I20260926 02:01:08.503377 30618 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:08.503433 30618 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:08.503644 30618 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:08.505255 30618 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data/instance:
uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.505717 30618 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/instance:
uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.511162 30618 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 02:01:08.513339 30632 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.514441 30618 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:08.514557 30618 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal
uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.514654 30618 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:08.526551 30618 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:08.527154 30618 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:08.527341 30618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:08.527958 30618 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 02:01:08.529022 30618 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 02:01:08.529070 30618 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.529132 30618 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 02:01:08.529165 30618 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.538750 30618 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.129:38981
I20260926 02:01:08.538774 30745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.129:38981 every 8 connection(s)
I20260926 02:01:08.539737 30618 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/data/info.pb
I20260926 02:01:08.543910 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30618
I20260926 02:01:08.544025 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/instance
I20260926 02:01:08.548060 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.89.130:0
--local_ip_for_outbound_sockets=127.29.89.130
--webserver_interface=127.29.89.130
--webserver_port=0
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 02:01:08.549721 30748 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:08.549957 30748 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.550287 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:08.550376 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.550581 30748 heartbeater.cc:507] Master 127.29.89.189:33299 requested a full tablet report, sending...
I20260926 02:01:08.550704 30747 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:08.550770 30747 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.550918 30747 heartbeater.cc:507] Master 127.29.89.190:45707 requested a full tablet report, sending...
I20260926 02:01:08.550581 30746 heartbeater.cc:507] Master 127.29.89.188:46477 requested a full tablet report, sending...
I20260926 02:01:08.552212 30510 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:08.552208 30438 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:08.552369 30368 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:08.553632 30510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.29.89.129:36447
W20260926 02:01:08.652266 30754 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:08.652462 30754 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:08.652508 30754 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:08.655725 30754 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:08.655829 30754 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.89.130
I20260926 02:01:08.659570 30754 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.89.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.89.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--never_fsync=true
--heap_profile_path=/tmp/kudu.30754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.89.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:08.660552 30754 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:08.661906 30754 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:08.668911 30759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:08.668937 30762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:08.669001 30754 server_base.cc:1061] running on GCE node
W20260926 02:01:08.668954 30760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:08.669479 30754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:08.669952 30754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:08.671100 30754 hybrid_clock.cc:648] HybridClock initialized: now 1790388068671090 us; error 24 us; skew 500 ppm
I20260926 02:01:08.672780 30754 webserver.cc:533] Webserver started at http://127.29.89.130:36779/ using document root <none> and password file <none>
I20260926 02:01:08.673417 30754 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:08.673501 30754 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:08.673720 30754 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:08.675274 30754 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data/instance:
uuid: "08fbed94c9e248d0a89843d39b01ef07"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.675752 30754 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/instance:
uuid: "08fbed94c9e248d0a89843d39b01ef07"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.679056 30754 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 02:01:08.681093 30768 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.682042 30754 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 02:01:08.682180 30754 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal
uuid: "08fbed94c9e248d0a89843d39b01ef07"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.682278 30754 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:08.692569 30754 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:08.693236 30754 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:08.693434 30754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:08.693992 30754 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 02:01:08.695065 30754 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 02:01:08.695138 30754 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.695209 30754 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 02:01:08.695243 30754 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.705713 30754 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.130:38777
I20260926 02:01:08.705782 30881 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.130:38777 every 8 connection(s)
I20260926 02:01:08.706822 30754 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/data/info.pb
I20260926 02:01:08.707209 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30754
I20260926 02:01:08.707324 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/instance
I20260926 02:01:08.711365 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.89.131:0
--local_ip_for_outbound_sockets=127.29.89.131
--webserver_interface=127.29.89.131
--webserver_port=0
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 02:01:08.718271 30884 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:08.718359 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:08.718484 30883 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:08.718546 30883 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.718549 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.718550 30884 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.719137 30882 heartbeater.cc:507] Master 127.29.89.188:46477 requested a full tablet report, sending...
I20260926 02:01:08.719137 30883 heartbeater.cc:507] Master 127.29.89.190:45707 requested a full tablet report, sending...
I20260926 02:01:08.719354 30884 heartbeater.cc:507] Master 127.29.89.189:33299 requested a full tablet report, sending...
I20260926 02:01:08.720319 30510 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:08.720357 30438 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:08.720433 30368 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:08.721012 30510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.29.89.130:40011
W20260926 02:01:08.815936 30889 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:08.816160 30889 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:08.816255 30889 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:08.819706 30889 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:08.819846 30889 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.89.131
I20260926 02:01:08.823751 30889 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.89.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.89.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--never_fsync=true
--heap_profile_path=/tmp/kudu.30889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.89.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:08.824713 30889 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:08.825827 30889 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:08.832500 30894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:08.832597 30897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:08.832502 30895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:08.832815 30889 server_base.cc:1061] running on GCE node
I20260926 02:01:08.833180 30889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:08.833314 30899 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b285db155484443594c9385155358d4d)
I20260926 02:01:08.833460 30899 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:08.833847 30889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:08.834467 30899 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:08.834904 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:08.835035 30889 hybrid_clock.cc:648] HybridClock initialized: now 1790388068835004 us; error 32 us; skew 500 ppm
I20260926 02:01:08.835063 30520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d" is_pre_election: true
I20260926 02:01:08.835732 30423 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:08.835953 30899 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 02:01:08.836853 30889 webserver.cc:533] Webserver started at http://127.29.89.131:33031/ using document root <none> and password file <none>
I20260926 02:01:08.837469 30889 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:08.837548 30889 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:08.837752 30889 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 02:01:08.839277 30889 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data/instance:
uuid: "02a024f3eff64b34b0fa7e46d6274abe"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.839740 30889 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/instance:
uuid: "02a024f3eff64b34b0fa7e46d6274abe"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.842929 30889 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 02:01:08.845074 30904 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.846040 30889 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:08.846164 30889 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal
uuid: "02a024f3eff64b34b0fa7e46d6274abe"
format_stamp: "Formatted at 2026-09-26 02:01:08 on dist-test-slave-1hsp"
I20260926 02:01:08.846261 30889 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:08.871798 30889 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:08.872407 30889 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:08.872587 30889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:08.873154 30889 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 02:01:08.874121 30889 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 02:01:08.874203 30889 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:08.874292 30889 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 02:01:08.874372 30889 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 02:01:08.883635 30889 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.131:36271
I20260926 02:01:08.883666 31017 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.131:36271 every 8 connection(s)
I20260926 02:01:08.884544 30889 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/data/info.pb
I20260926 02:01:08.890574 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 30889
I20260926 02:01:08.890687 30054 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/instance
I20260926 02:01:08.894237 31020 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:08.894342 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:08.894484 31020 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.894505 31019 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:08.894650 31019 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.894480 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:08.895103 31020 heartbeater.cc:507] Master 127.29.89.189:33299 requested a full tablet report, sending...
I20260926 02:01:08.895298 31018 heartbeater.cc:507] Master 127.29.89.188:46477 requested a full tablet report, sending...
I20260926 02:01:08.895103 31019 heartbeater.cc:507] Master 127.29.89.190:45707 requested a full tablet report, sending...
I20260926 02:01:08.896129 30438 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:08.896185 30510 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:08.896863 30368 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:08.896858 30510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.29.89.131:41181
I20260926 02:01:08.910645 30054 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260926 02:01:08.926614 30054 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu with pid 30477
W20260926 02:01:08.933931 31025 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:08.935379 31025 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.188:46477)
I20260926 02:01:09.055261 30746 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
W20260926 02:01:09.056156 30746 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.073765 30585 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:09.073864 30585 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: loading cluster ID for follower catalog manager: success
I20260926 02:01:09.075525 30585 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: acquiring CA information for follower catalog manager: success
I20260926 02:01:09.076545 30585 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:09.151242 30604 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:09.151350 30604 catalog_manager.cc:1601] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: loading cluster ID for follower catalog manager: success
I20260926 02:01:09.153308 30604 catalog_manager.cc:1623] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: acquiring CA information for follower catalog manager: success
I20260926 02:01:09.154448 30604 catalog_manager.cc:1651] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:09.222422 30882 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
W20260926 02:01:09.223392 30882 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.223906 31066 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b285db155484443594c9385155358d4d)
I20260926 02:01:09.224011 31066 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.224888 31066 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:09.225375 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
W20260926 02:01:09.226207 30352 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.226287 30352 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:09.226547 31066 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 02:01:09.398913 31018 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
W20260926 02:01:09.399924 31018 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.400429 31070 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b285db155484443594c9385155358d4d)
I20260926 02:01:09.400521 31070 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.400936 31070 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:09.401259 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:09.401381 30378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 1.
I20260926 02:01:09.401795 30423 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:09.401955 31070 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 02:01:09.402024 31070 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b285db155484443594c9385155358d4d)
I20260926 02:01:09.402076 31070 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 1 FOLLOWER]: Advancing to term 2
W20260926 02:01:09.402413 30423 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.403083 31070 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.403437 31070 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:09.403903 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:09.404039 30378 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 1 FOLLOWER]: Advancing to term 2
W20260926 02:01:09.404507 30423 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:09.404810 30378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 2.
I20260926 02:01:09.405097 30423 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters: b285db155484443594c9385155358d4d
I20260926 02:01:09.405306 31070 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 02:01:09.405514 31070 raft_consensus.cc:697] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 LEADER]: Becoming Leader. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Running, Role: LEADER
I20260926 02:01:09.405856 31070 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.407223 31073 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 2 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:09.407351 31073 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:09.407725 31075 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:09.408535 31075 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:09.408959 31075 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:09.409008 31075 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:09.409562 31075 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:09.409962 31075 catalog_manager.cc:6230] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Loaded TSK: 0
I20260926 02:01:09.410614 31075 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:09.411015 31075 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:09.411350 31075 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 02:01:09.414583 30438 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_44705ce850b5424984beb85e0103c16c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414849 30435 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414872 30433 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414692 30436 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414983 30434 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414664 30437 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414585 30439 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.415009 30430 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414955 30431 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_aba582f363ee48b38f7b72fe1507952f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:09.414778 30432 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40936:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 02:01:09.453166 30438 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_44705ce850b5424984beb85e0103c16c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 02:01:09.572508 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=36293
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:09.715600 31078 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:09.715975 31078 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:09.716135 31078 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:09.720841 31078 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:09.720981 31078 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:09.721066 31078 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:09.721143 31078 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:09.726697 31078 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=36293
--never_fsync=true
--heap_profile_path=/tmp/kudu.31078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:09.728150 31078 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:09.729233 31078 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:09.734544 31087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:09.734921 31084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:09.735222 31085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:09.736129 31078 server_base.cc:1061] running on GCE node
I20260926 02:01:09.736598 31078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:09.737454 31078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:09.738605 31078 hybrid_clock.cc:648] HybridClock initialized: now 1790388069738591 us; error 29 us; skew 500 ppm
I20260926 02:01:09.740348 31078 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:09.740872 31078 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:09.740967 31078 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:09.744236 31078 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:09.751668 31093 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:09.752905 31078 fs_manager.cc:730] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260926 02:01:09.753008 31078 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:09.753465 31078 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:09.784904 31078 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:09.785794 31078 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:09.786027 31078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:09.798522 31078 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:09.799984 31078 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:09.802500 31145 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:09.802773 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31078
I20260926 02:01:09.807775 31146 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:09.809958 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:09.817124 31146 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:09.824656 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:09.825312 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap complete.
I20260926 02:01:09.829290 31146 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.829829 31146 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:09.830590 31146 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.831916 31146 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:09.840405 31164 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:09.840590 31164 catalog_manager.cc:1601] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: success
I20260926 02:01:09.842955 31164 catalog_manager.cc:1623] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: acquiring CA information for follower catalog manager: success
I20260926 02:01:09.857144 31164 catalog_manager.cc:1651] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:09.843206 31150 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:09.857443 31150 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:09.916697 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:09.916872 31018 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
I20260926 02:01:09.916862 31171 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:09.916990 31171 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.917497 31171 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:09.917845 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
I20260926 02:01:09.918493 31102 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:46377; Asking this server to re-register.
I20260926 02:01:09.919260 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:09.920629 31102 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:09.932859 31120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d" is_pre_election: true
I20260926 02:01:09.933292 31120 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 995d22fd6e1b4cdeb3b8aa4c61b56272 in term 1.
I20260926 02:01:09.934557 30352 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:09.934849 31171 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 02:01:09.934903 31171 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:09.934949 31171 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 2 FOLLOWER]: Advancing to term 3
I20260926 02:01:09.936175 31171 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.936657 31171 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 3 election: Requested vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:09.937211 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2"
I20260926 02:01:09.938268 31120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d"
I20260926 02:01:09.938448 31120 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 1 FOLLOWER]: Advancing to term 3
I20260926 02:01:09.941511 31120 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 995d22fd6e1b4cdeb3b8aa4c61b56272 in term 3.
I20260926 02:01:09.941936 30352 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:09.942241 31171 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 02:01:09.942467 31171 raft_consensus.cc:697] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 LEADER]: Becoming Leader. State: Replica: 995d22fd6e1b4cdeb3b8aa4c61b56272, State: Running, Role: LEADER
I20260926 02:01:09.942838 31171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:09.944267 31171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995d22fd6e1b4cdeb3b8aa4c61b56272. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:09.944502 31171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:09.947703 31175 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:09.949512 31175 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:09.950098 31175 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:09.950238 31175 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:09.950944 31175 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:09.951447 31175 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Loaded TSK: 0
I20260926 02:01:09.952250 31175 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:09.952737 31175 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:09.953158 31175 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 02:01:09.971247 30378 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 LEADER]: Rejecting Update request from peer d95dbe14730b406b8f4ffced8abe38f2 for earlier term 2. Current term is 3. Ops: []
I20260926 02:01:09.972023 30438 catalog_manager.cc:2213] created HMS catalog entry for table [id=fdf617a170f34b4a9603645c9bebb4c0]
I20260926 02:01:09.973284 31176 consensus_queue.cc:1059] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 02:01:09.973660 31176 consensus_queue.cc:1243] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Peer 995d22fd6e1b4cdeb3b8aa4c61b56272 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260926 02:01:09.973927 31176 raft_consensus.cc:3055] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260926 02:01:09.974041 31176 raft_consensus.cc:740] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Running, Role: LEADER
I20260926 02:01:09.974344 31176 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
W20260926 02:01:09.974555 30437 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_3e297a6e8a69461b8c6d85132495df8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 02:01:09.974679 31176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 2 FOLLOWER]: Advancing to term 3
W20260926 02:01:09.975418 30439 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_095de6750e1242cfb2364d77e2b4deb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.976075 30436 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8917cd5381bb470e8ee76b412f3ec83a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.976541 30438 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
W20260926 02:01:09.976716 30430 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_351548b11fca4255832809bbaf8f8a19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.977353 30433 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fc856879f4024619ba0ceaa95ae58ad5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.977993 30435 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5f1c6f611ee8464b89f6dcd96dad2fc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.978675 30434 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_28bfb85f5dab4406b28b8dc939814d7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.979329 30432 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_67533d3acf714eba8041a0a5675dd2eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 02:01:09.979597 30431 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_aba582f363ee48b38f7b72fe1507952f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 02:01:10.010994 30437 catalog_manager.cc:2213] created HMS catalog entry for table [id=e2951ee585ac456da08f6559d8b615c6]
W20260926 02:01:10.011721 30437 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.047749 30439 catalog_manager.cc:2213] created HMS catalog entry for table [id=b477be9c440047bcbbeac57cdc7897db]
W20260926 02:01:10.048383 30439 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.075594 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:10.075775 30746 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
I20260926 02:01:10.076495 31102 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:50013; Asking this server to re-register.
I20260926 02:01:10.077086 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:10.078001 31102 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:10.090870 30436 catalog_manager.cc:2213] created HMS catalog entry for table [id=4f58e9ae99704f108daa206ff9a6f562]
W20260926 02:01:10.091461 30436 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.109153 31120 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Refusing update from remote peer 995d22fd6e1b4cdeb3b8aa4c61b56272: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 02:01:10.109923 31173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 02:01:10.113765 31150 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995d22fd6e1b4cdeb3b8aa4c61b56272. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.113893 31150 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:10.114117 30449 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: Refusing update from remote peer 995d22fd6e1b4cdeb3b8aa4c61b56272: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 02:01:10.114669 31173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 02:01:10.114787 31173 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Peer d95dbe14730b406b8f4ffced8abe38f2 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260926 02:01:10.115454 30449 pending_rounds.cc:85] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Aborting all ops after (but not including) 4
I20260926 02:01:10.115540 30449 pending_rounds.cc:107] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260926 02:01:10.115583 30449 raft_consensus.cc:2889] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260926 02:01:10.116216 31178 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995d22fd6e1b4cdeb3b8aa4c61b56272. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.116324 31178 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:10.117616 31171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.117739 31171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:10.118409 31178 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.118559 31178 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:10.123059 31173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.123183 31173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:10.124313 31150 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:10.124449 31150 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:10.124887 31181 mvcc.cc:204] Tried to move back new op lower bound from 7333429535157514240 to 7333429534489960448. Current Snapshot: MvccSnapshot[applied={T|T < 7333429535157514240}]
I20260926 02:01:10.231679 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:10.231827 30882 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
I20260926 02:01:10.232791 31102 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:43315; Asking this server to re-register.
I20260926 02:01:10.233544 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:10.234426 31102 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:10.560983 30747 heartbeater.cc:499] Master 127.29.89.190:45707 was elected leader, sending a full tablet report...
I20260926 02:01:10.624244 31150 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:10.624400 31150 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:10.625967 31150 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:10.627125 30378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:10.627547 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
I20260926 02:01:10.628705 31094 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:10.629181 31150 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 02:01:10.650169 30438 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=fdf617a170f34b4a9603645c9bebb4c0]
I20260926 02:01:10.650707 30438 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 29) took 1236 ms. Trace:
I20260926 02:01:10.650789 30438 rpcz_store.cc:276] 0926 02:01:09.414228 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414329 (+ 101us) service_pool.cc:224] Handling call
0926 02:01:09.453424 (+ 39095us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.454468 (+ 1044us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:09.972015 (+517547us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:09.976603 (+ 4588us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:10.650684 (+674081us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:09.976073 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 02:01:09.976167 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0926 02:01:09.976173 (+ 6us) write_op.cc:435] Acquired schema lock
0926 02:01:09.976175 (+ 2us) tablet.cc:662] Decoding operations
0926 02:01:09.976256 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0926 02:01:09.976267 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0926 02:01:09.976270 (+ 3us) tablet.cc:685] Acquiring locks for 2 operations
0926 02:01:09.976323 (+ 53us) tablet.cc:701] Row locks acquired
0926 02:01:09.976324 (+ 1us) write_op.cc:260] PREPARE: finished
0926 02:01:09.976372 (+ 48us) op_driver.cc:389] HandleFailure(Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.)
0926 02:01:09.976382 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:09.976388 (+ 6us) write_op.cc:454] Released schema lock
0926 02:01:09.976392 (+ 4us) write_op.cc:335] FINISH: Op aborted
0926 02:01:09.976792 (+ 400us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:09.976810 (+ 18us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":115758,"Hive Metastore.run_cpu_time_us":662,"Hive Metastore.run_wall_time_us":1074706,"spinlock_wait_cycles":71936,"child_traces":[["op",{"num_ops":2,"prepare.queue_time_us":3011,"prepare.run_cpu_time_us":418,"prepare.run_wall_time_us":699,"thread_start_us":100,"threads_started":1}]]}
W20260926 02:01:10.651827 30438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.29.89.129:48585 (request call id 3) took 1094 ms (client timeout 1000 ms). Trace:
W20260926 02:01:10.651880 30438 rpcz_store.cc:269] 0926 02:01:09.557491 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:10.651702 (+1094211us) service_pool.cc:211] Skipping call since client already timed out
0926 02:01:10.651801 (+ 99us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260926 02:01:10.652279 30438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.29.89.130:49121 (request call id 3) took 928 ms (client timeout 1000 ms). Trace:
W20260926 02:01:10.652333 30438 rpcz_store.cc:269] 0926 02:01:09.723342 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:10.652007 (+928665us) service_pool.cc:224] Handling call
0926 02:01:10.652268 (+ 261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260926 02:01:10.652621 30438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.29.89.131:39595 (request call id 3) took 753 ms (client timeout 1000 ms). Trace:
W20260926 02:01:10.652666 30438 rpcz_store.cc:269] 0926 02:01:09.899302 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:10.652453 (+753151us) service_pool.cc:224] Handling call
0926 02:01:10.652612 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260926 02:01:10.653827 31026 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.189:33299)
I20260926 02:01:10.656282 31078 master_runner.cc:233] Existing masters have differing terms: 2 vs 3
I20260926 02:01:10.656405 31078 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:10.657209 30368 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42946:
name: "default.Table_44705ce850b5424984beb85e0103c16c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:10.662113 30584 catalog_manager.cc:2811] Deleting table default.table_44705ce850b5424984beb85e0103c16c [id=fdf617a170f34b4a9603645c9bebb4c0] in response to Hive Metastore notification log event 5
W20260926 02:01:10.662448 30584 hms_notification_log_listener.cc:329] Failed to handle Hive Metastore notification: 5 DROP_TABLE default.table_44705ce850b5424984beb85e0103c16c: Not found: the table does not exist: table_id: "fdf617a170f34b4a9603645c9bebb4c0" table_name: "default.table_44705ce850b5424984beb85e0103c16c"
W20260926 02:01:10.671852 30368 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_44705ce850b5424984beb85e0103c16c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 02:01:10.725283 30430 catalog_manager.cc:2213] created HMS catalog entry for table [id=d6b5e0e0086e42698f03eb96e3becabb]
W20260926 02:01:10.725958 30430 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.726809 30883 heartbeater.cc:499] Master 127.29.89.190:45707 was elected leader, sending a full tablet report...
I20260926 02:01:10.736943 30368 catalog_manager.cc:2213] created HMS catalog entry for table [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:10.749244 30368 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42946:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
F20260926 02:01:10.754468 30584 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 2 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fc1fb93bd5d google::LogMessage::Fail() at ??:0
@ 0x7fc1fb93fb23 google::LogMessage::SendToLog() at ??:0
@ 0x7fc1fb93b75c google::LogMessage::Flush() at ??:0
@ 0x7fc1fb93b9a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fc200efc959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 02:01:10.780192 30816 tablet_service.cc:1511] Processing CreateTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.780949 30680 tablet_service.cc:1511] Processing CreateTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.781279 30816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee779b26708f4c5e95b2a007ff0ccb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:10.781634 30680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee779b26708f4c5e95b2a007ff0ccb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:10.781953 30952 tablet_service.cc:1511] Processing CreateTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.782837 30952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee779b26708f4c5e95b2a007ff0ccb5e. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fc200efb870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 02:01:10.792047 30367 master_service.cc:960] ReplaceTablet: received request to replace tablet ee779b26708f4c5e95b2a007ff0ccb5e from {username='slave'} at 127.0.0.1:42946
I20260926 02:01:10.792383 30363 master_service.cc:960] ReplaceTablet: received request to replace tablet ee779b26708f4c5e95b2a007ff0ccb5e from {username='slave'} at 127.0.0.1:42946
I20260926 02:01:10.793144 30366 master_service.cc:960] ReplaceTablet: received request to replace tablet ee779b26708f4c5e95b2a007ff0ccb5e from {username='slave'} at 127.0.0.1:42946
I20260926 02:01:10.793316 30365 master_service.cc:960] ReplaceTablet: received request to replace tablet ee779b26708f4c5e95b2a007ff0ccb5e from {username='slave'} at 127.0.0.1:42946
I20260926 02:01:10.793548 30364 master_service.cc:960] ReplaceTablet: received request to replace tablet ee779b26708f4c5e95b2a007ff0ccb5e from {username='slave'} at 127.0.0.1:42946
@ 0x7fc200efafd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
I20260926 02:01:10.799526 30367 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Sending DeleteTablet for 3 replicas of tablet ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:10.799921 30367 catalog_manager.cc:6934] ReplaceTablet: tablet ee779b26708f4c5e95b2a007ff0ccb5e deleted and replaced by tablet d63b60f303f74dca95eebb234fcb6316
I20260926 02:01:10.801612 30816 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:33934
I20260926 02:01:10.802716 31205 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
I20260926 02:01:10.805140 31205 tablet_bootstrap.cc:654] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.806083 31205 log.cc:826] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:10.806711 31206 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
I20260926 02:01:10.808918 31206 tablet_bootstrap.cc:654] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.809818 31206 log.cc:826] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Log is configured to *not* fsync() on all Append() calls
@ 0x7fc200f006da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
W20260926 02:01:10.811208 30355 catalog_manager.cc:5043] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): delete failed for tablet ee779b26708f4c5e95b2a007ff0ccb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee779b26708f4c5e95b2a007ff0ccb5e already in progress: creating tablet
I20260926 02:01:10.812328 31204 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:10.813289 30816 tablet_service.cc:1511] Processing CreateTablet for tablet d63b60f303f74dca95eebb234fcb6316 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.813552 30816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d63b60f303f74dca95eebb234fcb6316. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x55e638e68f7e std::function<>::operator()() at ??:0
I20260926 02:01:10.816510 30951 tablet_service.cc:1511] Processing CreateTablet for tablet d63b60f303f74dca95eebb234fcb6316 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.816841 30951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d63b60f303f74dca95eebb234fcb6316. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fc1fce17c6a kudu::Thread::SuperviseThread() at ??:0
I20260926 02:01:10.818610 31204 tablet_bootstrap.cc:654] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.819525 31204 log.cc:826] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:10.819633 30679 tablet_service.cc:1511] Processing CreateTablet for tablet d63b60f303f74dca95eebb234fcb6316 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:10.819906 30679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d63b60f303f74dca95eebb234fcb6316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:10.821372 31206 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:10.821592 31206 ts_tablet_manager.cc:1403] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260926 02:01:10.826102 31205 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:10.826292 31205 ts_tablet_manager.cc:1403] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260926 02:01:10.829787 31205 raft_consensus.cc:359] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.830005 31205 raft_consensus.cc:385] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.830065 31205 raft_consensus.cc:740] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.830488 30952 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:35368
I20260926 02:01:10.830603 31205 consensus_queue.cc:260] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.830948 30680 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:38884
I20260926 02:01:10.831517 31205 ts_tablet_manager.cc:1434] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260926 02:01:10.832607 30433 catalog_manager.cc:2213] created HMS catalog entry for table [id=268787de102d486f9eee386d9b8bbad1]
W20260926 02:01:10.833139 30433 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.837590 30816 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:33934
I20260926 02:01:10.837970 31207 tablet_replica.cc:333] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: stopping tablet replica
I20260926 02:01:10.838222 31207 raft_consensus.cc:2243] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:10.838477 31207 raft_consensus.cc:2272] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:10.839341 31207 ts_tablet_manager.cc:1916] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:10.841746 31207 ts_tablet_manager.cc:1929] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 02:01:10.841823 31207 log.cc:1199] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/wals/ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:10.842088 31207 ts_tablet_manager.cc:1950] T ee779b26708f4c5e95b2a007ff0ccb5e P 08fbed94c9e248d0a89843d39b01ef07: Deleting consensus metadata
I20260926 02:01:10.842664 30355 catalog_manager.cc:5063] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): tablet ee779b26708f4c5e95b2a007ff0ccb5e (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
W20260926 02:01:10.843936 30354 catalog_manager.cc:5043] TS 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): delete failed for tablet ee779b26708f4c5e95b2a007ff0ccb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee779b26708f4c5e95b2a007ff0ccb5e already in progress: creating tablet
I20260926 02:01:10.844304 31205 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
W20260926 02:01:10.844811 30353 catalog_manager.cc:5043] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): delete failed for tablet ee779b26708f4c5e95b2a007ff0ccb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee779b26708f4c5e95b2a007ff0ccb5e already in progress: creating tablet
@ 0x7fc1fd7966db start_thread at ??:0
I20260926 02:01:10.845662 31205 tablet_bootstrap.cc:654] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.846614 31206 raft_consensus.cc:359] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.846882 31206 raft_consensus.cc:385] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.846943 31206 raft_consensus.cc:740] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.847448 31206 consensus_queue.cc:260] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
@ 0x7fc1faaf261f clone at ??:0
I20260926 02:01:10.848708 31206 ts_tablet_manager.cc:1434] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260926 02:01:10.849819 31205 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:10.849905 31205 ts_tablet_manager.cc:1403] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260926 02:01:10.850334 31205 raft_consensus.cc:359] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.850432 31205 raft_consensus.cc:385] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.850431 31204 tablet_bootstrap.cc:492] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
I20260926 02:01:10.850461 31205 raft_consensus.cc:740] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.850651 31204 ts_tablet_manager.cc:1403] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.000s
I20260926 02:01:10.853466 31205 consensus_queue.cc:260] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.853995 31204 raft_consensus.cc:359] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.855672 31204 raft_consensus.cc:385] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.855737 31204 raft_consensus.cc:740] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.855921 31205 ts_tablet_manager.cc:1434] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260926 02:01:10.859619 31019 heartbeater.cc:499] Master 127.29.89.190:45707 was elected leader, sending a full tablet report...
I20260926 02:01:10.860654 31206 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
I20260926 02:01:10.861765 31206 tablet_bootstrap.cc:654] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.862691 31204 consensus_queue.cc:260] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.863602 31204 ts_tablet_manager.cc:1434] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260926 02:01:10.866600 31204 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:10.867602 31204 tablet_bootstrap.cc:654] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:10.868773 31204 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
I20260926 02:01:10.868851 31204 ts_tablet_manager.cc:1403] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:10.869400 31204 raft_consensus.cc:359] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.869505 31204 raft_consensus.cc:385] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.869540 31204 raft_consensus.cc:740] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.869681 31204 consensus_queue.cc:260] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.870036 31204 ts_tablet_manager.cc:1434] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 02:01:10.870730 31206 tablet_bootstrap.cc:492] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:10.870816 31206 ts_tablet_manager.cc:1403] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260926 02:01:10.871387 31206 raft_consensus.cc:359] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.871492 31206 raft_consensus.cc:385] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:10.871527 31206 raft_consensus.cc:740] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:10.871660 31206 consensus_queue.cc:260] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.871933 31206 ts_tablet_manager.cc:1434] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260926 02:01:10.887039 31021 tablet.cc:2433] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 02:01:10.898550 30435 catalog_manager.cc:2213] created HMS catalog entry for table [id=01eeb1c1fb824b99b09e00c9b9ab806e]
W20260926 02:01:10.899169 30435 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:10.941044 31216 raft_consensus.cc:493] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:10.941226 31216 raft_consensus.cc:515] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.942687 31216 leader_election.cc:290] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:10.948390 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d63b60f303f74dca95eebb234fcb6316" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07" is_pre_election: true
I20260926 02:01:10.948673 30836 raft_consensus.cc:2468] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:10.949173 30908 leader_election.cc:304] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 08fbed94c9e248d0a89843d39b01ef07; no voters:
I20260926 02:01:10.949489 31216 raft_consensus.cc:2804] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:10.949584 31216 raft_consensus.cc:493] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:10.949636 31216 raft_consensus.cc:3060] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:10.951229 31216 raft_consensus.cc:515] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.951603 31216 leader_election.cc:290] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Requested vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:10.952059 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d63b60f303f74dca95eebb234fcb6316" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07"
I20260926 02:01:10.952183 30836 raft_consensus.cc:3060] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:10.953260 30836 raft_consensus.cc:2468] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:10.953346 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d63b60f303f74dca95eebb234fcb6316" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" is_pre_election: true
I20260926 02:01:10.953579 30700 raft_consensus.cc:2468] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:10.953603 30908 leader_election.cc:304] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 08fbed94c9e248d0a89843d39b01ef07; no voters:
I20260926 02:01:10.953783 31216 raft_consensus.cc:2804] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:10.953851 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d63b60f303f74dca95eebb234fcb6316" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
I20260926 02:01:10.953935 30699 raft_consensus.cc:3060] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:10.953969 31216 raft_consensus.cc:697] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Becoming Leader. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Running, Role: LEADER
I20260926 02:01:10.954339 31216 consensus_queue.cc:237] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:10.955473 30699 raft_consensus.cc:2468] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:10.956506 30434 catalog_manager.cc:2213] created HMS catalog entry for table [id=6d4be6091284480ba5718a9d15ebabb6]
W20260926 02:01:10.957048 30434 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
W20260926 02:01:10.962349 30885 tablet.cc:2433] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 02:01:10.966810 31232 raft_consensus.cc:493] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:10.966974 31232 raft_consensus.cc:515] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.967722 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee779b26708f4c5e95b2a007ff0ccb5e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" is_pre_election: true
I20260926 02:01:10.967845 30699 raft_consensus.cc:2468] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:10.968231 30907 leader_election.cc:304] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:10.968418 31216 raft_consensus.cc:2804] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:10.968492 31216 raft_consensus.cc:493] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:10.968542 31216 raft_consensus.cc:3060] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
W20260926 02:01:10.969354 30908 leader_election.cc:343] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:10.969549 31232 leader_election.cc:290] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:10.969594 31216 raft_consensus.cc:515] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
I20260926 02:01:10.969973 31216 leader_election.cc:290] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Requested vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:10.970268 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee779b26708f4c5e95b2a007ff0ccb5e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
I20260926 02:01:10.970404 30699 raft_consensus.cc:3060] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:10.971349 30699 raft_consensus.cc:2468] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:10.971700 30907 leader_election.cc:304] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:10.971876 31216 raft_consensus.cc:2804] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:10.971973 31216 raft_consensus.cc:697] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Becoming Leader. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Running, Role: LEADER
I20260926 02:01:10.972107 31216 consensus_queue.cc:237] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } }
W20260926 02:01:10.972543 30908 leader_election.cc:343] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:11.024528 30432 catalog_manager.cc:2213] created HMS catalog entry for table [id=c79138f00a124c90b674ad48ae7f2a19]
W20260926 02:01:11.025494 30432 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
W20260926 02:01:11.043780 30749 tablet.cc:2433] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 02:01:11.062127 30431 catalog_manager.cc:2213] created HMS catalog entry for table [id=640075642bc74658a5b1bf0ebf037db9]
W20260926 02:01:11.062836 30431 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER
I20260926 02:01:11.115054 30437 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=e2951ee585ac456da08f6559d8b615c6]
I20260926 02:01:11.115365 30437 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 31) took 1700 ms. Trace:
I20260926 02:01:11.115428 30437 rpcz_store.cc:276] 0926 02:01:09.414461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414577 (+ 116us) service_pool.cc:224] Handling call
0926 02:01:09.974663 (+560086us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.975039 (+ 376us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.010978 (+ 35939us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.011646 (+ 668us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.011648 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.011801 (+ 153us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.115340 (+1103539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.011593 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.011602 (+ 9us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.011608 (+ 6us) write_op.cc:454] Released schema lock
0926 02:01:10.011611 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1050177,"Hive Metastore.run_cpu_time_us":452,"Hive Metastore.run_wall_time_us":88752,"child_traces":[["op",{}]]}
I20260926 02:01:11.166018 30439 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=b477be9c440047bcbbeac57cdc7897db]
I20260926 02:01:11.166359 30439 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 30) took 1752 ms. Trace:
I20260926 02:01:11.166421 30439 rpcz_store.cc:276] 0926 02:01:09.414291 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414377 (+ 86us) service_pool.cc:224] Handling call
0926 02:01:09.975499 (+561122us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.975818 (+ 319us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.047737 (+ 71919us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.048334 (+ 597us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.048335 (+ 1us) write_op.cc:454] Released schema lock
0926 02:01:10.048437 (+ 102us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.166337 (+1117900us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.048280 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.048294 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.048298 (+ 4us) write_op.cc:454] Released schema lock
0926 02:01:10.048301 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1101681,"Hive Metastore.run_cpu_time_us":448,"Hive Metastore.run_wall_time_us":87582,"child_traces":[["op",{}]]}
I20260926 02:01:11.211519 30436 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=4f58e9ae99704f108daa206ff9a6f562]
I20260926 02:01:11.211820 30436 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 32) took 1797 ms. Trace:
I20260926 02:01:11.211866 30436 rpcz_store.cc:276] 0926 02:01:09.414523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414615 (+ 92us) service_pool.cc:224] Handling call
0926 02:01:09.976153 (+561538us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.976434 (+ 281us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.090857 (+114423us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.091405 (+ 548us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.091407 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.091533 (+ 126us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.211799 (+1120266us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.091352 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.091361 (+ 9us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.091365 (+ 4us) write_op.cc:454] Released schema lock
0926 02:01:10.091369 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1145651,"Hive Metastore.run_cpu_time_us":454,"Hive Metastore.run_wall_time_us":88521,"mutex_wait_us":114,"child_traces":[["op",{}]]}
W20260926 02:01:11.237076 31025 connection.cc:573] client connection to 127.29.89.190:45707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.237463 31025 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.190:45707)
W20260926 02:01:11.237622 31025 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.190:45707)
W20260926 02:01:11.238210 30905 connection.cc:573] client connection to 127.29.89.190:45707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.240347 30769 connection.cc:573] client connection to 127.29.89.190:45707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.243336 30633 connection.cc:573] client connection to 127.29.89.190:45707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.244338 31230 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 02:01:11.238903 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.29.89.190:45707 (local address 127.29.89.131:35519)
0926 02:01:11.239124 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 02:01:11.239139 (+ 15us) client_negotiation.cc:175] Beginning negotiation
0926 02:01:11.239244 (+ 105us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 02:01:11.243747 (+ 4503us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":134}
W20260926 02:01:11.246104 31236 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 02:01:11.242491 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.29.89.190:45707 (local address 127.29.89.130:36311)
0926 02:01:11.245541 (+ 3050us) negotiation.cc:107] Waiting for socket to connect
0926 02:01:11.245591 (+ 50us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":2952,"thread_start_us":128,"threads_started":1}
W20260926 02:01:11.247241 31235 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 02:01:11.239698 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.29.89.190:45707 (local address 127.0.0.1:41144)
0926 02:01:11.246514 (+ 6816us) negotiation.cc:107] Waiting for socket to connect
0926 02:01:11.246570 (+ 56us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":6712,"thread_start_us":73,"threads_started":1}
I20260926 02:01:11.279577 30430 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=d6b5e0e0086e42698f03eb96e3becabb]
I20260926 02:01:11.279856 30430 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 38) took 1864 ms. Trace:
I20260926 02:01:11.279922 30430 rpcz_store.cc:276] 0926 02:01:09.414890 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414937 (+ 47us) service_pool.cc:224] Handling call
0926 02:01:09.976807 (+561870us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.977089 (+ 282us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.725270 (+748181us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.725899 (+ 629us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.725901 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.726035 (+ 134us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.279842 (+553807us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.725847 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.725855 (+ 8us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.725860 (+ 5us) write_op.cc:454] Released schema lock
0926 02:01:10.725863 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1158426,"Hive Metastore.run_cpu_time_us":431,"Hive Metastore.run_wall_time_us":143053,"spinlock_wait_cycles":5888,"child_traces":[["op",{}]]}
I20260926 02:01:11.324261 31238 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:11.324380 31238 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:11.324986 31238 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:11.325501 31240 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:11.325609 31240 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:11.325703 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
W20260926 02:01:11.325660 31094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:11.325831 30449 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 3.
I20260926 02:01:11.326057 31240 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:11.326176 31095 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:11.326406 31238 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 02:01:11.326470 31238 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:11.326526 31238 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 3 FOLLOWER]: Advancing to term 4
I20260926 02:01:11.327782 31238 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:11.328334 31238 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:11.329380 31094 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:11.329556 30449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2"
I20260926 02:01:11.329689 30449 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 3 FOLLOWER]: Advancing to term 4
W20260926 02:01:11.330040 31094 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:11.330583 30423 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:11.330860 30449 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 4.
I20260926 02:01:11.331328 31095 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272
I20260926 02:01:11.331501 31238 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Leader election won for term 4
I20260926 02:01:11.331678 31238 raft_consensus.cc:697] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 LEADER]: Becoming Leader. State: Replica: b285db155484443594c9385155358d4d, State: Running, Role: LEADER
I20260926 02:01:11.332007 31238 consensus_queue.cc:237] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:11.332358 31120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d" is_pre_election: true
I20260926 02:01:11.332780 30423 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:11.332947 31240 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 02:01:11.333276 31246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 4 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:11.333380 31246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:11.333735 31247 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:11.335564 31247 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:11.336069 30433 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=268787de102d486f9eee386d9b8bbad1]
I20260926 02:01:11.336272 30433 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 35) took 1921 ms. Trace:
I20260926 02:01:11.336321 30433 rpcz_store.cc:276] 0926 02:01:09.414715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414818 (+ 103us) service_pool.cc:224] Handling call
0926 02:01:09.977431 (+562613us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.977720 (+ 289us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.832598 (+854878us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.833088 (+ 490us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.833090 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.833201 (+ 111us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.336258 (+503057us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.833041 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.833049 (+ 8us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.833053 (+ 4us) write_op.cc:454] Released schema lock
0926 02:01:10.833056 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1193795,"Hive Metastore.run_cpu_time_us":387,"Hive Metastore.run_wall_time_us":163664,"mutex_wait_us":133,"child_traces":[["op",{}]]}
I20260926 02:01:11.338049 31247 tablet_loader.cc:96] loaded metadata for tablet d63b60f303f74dca95eebb234fcb6316 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:11.338518 31247 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:11.339066 31247 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:11.339146 31247 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:11.339684 31247 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:11.340088 31247 catalog_manager.cc:6230] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Loaded TSK: 0
I20260926 02:01:11.340848 31247 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:11.341328 31247 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:11.341786 31247 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:11.348068 31101 master_service.cc:960] ReplaceTablet: received request to replace tablet d63b60f303f74dca95eebb234fcb6316 from {username='slave'} at 127.0.0.1:53234
I20260926 02:01:11.348397 31105 master_service.cc:960] ReplaceTablet: received request to replace tablet d63b60f303f74dca95eebb234fcb6316 from {username='slave'} at 127.0.0.1:53234
I20260926 02:01:11.348670 31107 master_service.cc:960] ReplaceTablet: received request to replace tablet d63b60f303f74dca95eebb234fcb6316 from {username='slave'} at 127.0.0.1:53234
I20260926 02:01:11.348882 31109 master_service.cc:960] ReplaceTablet: received request to replace tablet d63b60f303f74dca95eebb234fcb6316 from {username='slave'} at 127.0.0.1:53234
I20260926 02:01:11.348979 31110 master_service.cc:960] ReplaceTablet: received request to replace tablet d63b60f303f74dca95eebb234fcb6316 from {username='slave'} at 127.0.0.1:53234
I20260926 02:01:11.347615 31102 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:11.348338 31104 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:11.348826 31108 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:11.348604 31106 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53234:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
I20260926 02:01:11.348209 31103 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:11.352437 30449 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Refusing update from remote peer b285db155484443594c9385155358d4d: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 4 index: 13. (index mismatch)
I20260926 02:01:11.352982 31246 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260926 02:01:11.355581 31094 consensus_peers.cc:597] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d -> Peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Couldn't send request to peer 995d22fd6e1b4cdeb3b8aa4c61b56272. Status: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 02:01:11.356079 31246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:11.356199 31246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:11.357906 31240 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 4 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:11.358018 31240 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:11.358111 31240 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:11.358171 31240 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:11.359735 31101 catalog_manager.cc:6133] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Sending DeleteTablet for 3 replicas of tablet d63b60f303f74dca95eebb234fcb6316
F20260926 02:01:11.360499 31163 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 12 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260926 02:01:11.362722 31110 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:11.363539 31109 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f39e587dd5d google::LogMessage::Fail() at ??:0
I20260926 02:01:11.366456 30680 tablet_service.cc:1558] Processing DeleteTablet for tablet d63b60f303f74dca95eebb234fcb6316 with delete_type TABLET_DATA_DELETED (replaced by tablet e9043632256f4e8e9087287de5118010) from {username='slave'} at 127.0.0.1:38896
@ 0x7f39e5881b23 google::LogMessage::SendToLog() at ??:0
I20260926 02:01:11.369756 30816 tablet_service.cc:1558] Processing DeleteTablet for tablet d63b60f303f74dca95eebb234fcb6316 with delete_type TABLET_DATA_DELETED (replaced by tablet e9043632256f4e8e9087287de5118010) from {username='slave'} at 127.0.0.1:33942
@ 0x7f39e587d75c google::LogMessage::Flush() at ??:0
I20260926 02:01:11.370704 31101 catalog_manager.cc:6934] ReplaceTablet: tablet d63b60f303f74dca95eebb234fcb6316 deleted and replaced by tablet e9043632256f4e8e9087287de5118010
@ 0x7f39e587d9a1 google::LogMessage::~LogMessage() at ??:0
I20260926 02:01:11.374440 30816 tablet_service.cc:1511] Processing CreateTablet for tablet e9043632256f4e8e9087287de5118010 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:11.374781 30816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9043632256f4e8e9087287de5118010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:11.379498 31255 tablet_replica.cc:333] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: stopping tablet replica
I20260926 02:01:11.379624 30952 tablet_service.cc:1558] Processing DeleteTablet for tablet d63b60f303f74dca95eebb234fcb6316 with delete_type TABLET_DATA_DELETED (replaced by tablet e9043632256f4e8e9087287de5118010) from {username='slave'} at 127.0.0.1:35372
I20260926 02:01:11.379776 30680 tablet_service.cc:1511] Processing CreateTablet for tablet e9043632256f4e8e9087287de5118010 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:11.380002 30951 tablet_service.cc:1511] Processing CreateTablet for tablet e9043632256f4e8e9087287de5118010 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:11.380072 30680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9043632256f4e8e9087287de5118010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:11.380234 30951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9043632256f4e8e9087287de5118010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:11.382076 31262 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
@ 0x7f39eae3e959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 02:01:11.383420 31262 tablet_bootstrap.cc:654] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:11.383812 31257 tablet_replica.cc:333] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: stopping tablet replica
I20260926 02:01:11.383935 31257 raft_consensus.cc:2243] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:11.384049 31257 raft_consensus.cc:2272] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:11.384447 31257 ts_tablet_manager.cc:1916] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:11.384585 31262 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:11.384670 31262 ts_tablet_manager.cc:1403] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 02:01:11.385247 31262 raft_consensus.cc:359] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.385366 31262 raft_consensus.cc:385] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:11.385406 31262 raft_consensus.cc:740] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:11.385568 31262 consensus_queue.cc:260] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.385906 31262 ts_tablet_manager.cc:1434] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:11.379819 31255 raft_consensus.cc:2243] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:11.386556 31255 raft_consensus.cc:2272] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:11.387833 31263 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:11.388114 31260 tablet_replica.cc:333] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: stopping tablet replica
I20260926 02:01:11.388422 31260 raft_consensus.cc:2243] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Raft consensus shutting down.
I20260926 02:01:11.388725 31260 pending_rounds.cc:70] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Trying to abort 1 pending ops.
I20260926 02:01:11.388821 31260 pending_rounds.cc:77] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333429538632306688 op_type: NO_OP noop_request { }
I20260926 02:01:11.388868 31263 tablet_bootstrap.cc:654] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:11.388913 31260 raft_consensus.cc:2889] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 02:01:11.388979 31260 raft_consensus.cc:2272] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:11.390239 31255 ts_tablet_manager.cc:1916] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:11.390417 31263 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
@ 0x7f39eae3d870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 02:01:11.390502 31263 ts_tablet_manager.cc:1403] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 02:01:11.391077 31263 raft_consensus.cc:359] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.391201 31263 raft_consensus.cc:385] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:11.391242 31263 raft_consensus.cc:740] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:11.391407 31263 consensus_queue.cc:260] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.391832 31263 ts_tablet_manager.cc:1434] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:11.392086 30746 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
@ 0x7f39eae3cfd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
I20260926 02:01:11.393913 31257 ts_tablet_manager.cc:1929] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 02:01:11.393985 31257 log.cc:1199] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/wals/d63b60f303f74dca95eebb234fcb6316
I20260926 02:01:11.394228 31257 ts_tablet_manager.cc:1950] T d63b60f303f74dca95eebb234fcb6316 P 08fbed94c9e248d0a89843d39b01ef07: Deleting consensus metadata
I20260926 02:01:11.395015 31259 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
I20260926 02:01:11.395172 31260 ts_tablet_manager.cc:1916] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:11.395161 30680 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:38896
I20260926 02:01:11.396049 31259 tablet_bootstrap.cc:654] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:11.397475 31260 ts_tablet_manager.cc:1929] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:11.397550 31260 log.cc:1199] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/wals/d63b60f303f74dca95eebb234fcb6316
I20260926 02:01:11.397866 31260 ts_tablet_manager.cc:1950] T d63b60f303f74dca95eebb234fcb6316 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting consensus metadata
I20260926 02:01:11.398267 31097 catalog_manager.cc:5063] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): tablet d63b60f303f74dca95eebb234fcb6316 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:11.399111 31095 catalog_manager.cc:5063] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): tablet d63b60f303f74dca95eebb234fcb6316 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:11.398902 31255 ts_tablet_manager.cc:1929] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 02:01:11.399319 31255 log.cc:1199] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/wals/d63b60f303f74dca95eebb234fcb6316
I20260926 02:01:11.399590 31255 ts_tablet_manager.cc:1950] T d63b60f303f74dca95eebb234fcb6316 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting consensus metadata
I20260926 02:01:11.400516 31255 tablet_replica.cc:333] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: stopping tablet replica
I20260926 02:01:11.400625 31255 raft_consensus.cc:2243] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:11.400723 31255 raft_consensus.cc:2272] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:11.400686 31096 catalog_manager.cc:5063] TS 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): tablet d63b60f303f74dca95eebb234fcb6316 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:11.401031 31255 ts_tablet_manager.cc:1916] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f39eae426da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55ba2215df7e std::function<>::operator()() at ??:0
I20260926 02:01:11.402916 31255 ts_tablet_manager.cc:1929] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 02:01:11.402976 31255 log.cc:1199] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/wals/ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:11.403192 31255 ts_tablet_manager.cc:1950] T ee779b26708f4c5e95b2a007ff0ccb5e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting consensus metadata
@ 0x7f39e6d59c6a kudu::Thread::SuperviseThread() at ??:0
I20260926 02:01:11.407137 31259 tablet_bootstrap.cc:492] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:11.407249 31259 ts_tablet_manager.cc:1403] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260926 02:01:11.407776 31259 raft_consensus.cc:359] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.407881 31259 raft_consensus.cc:385] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:11.407908 31259 raft_consensus.cc:740] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:11.408073 31259 consensus_queue.cc:260] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
@ 0x7f39e76d86db start_thread at ??:0
I20260926 02:01:11.408404 31259 ts_tablet_manager.cc:1434] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 02:01:11.408576 30882 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
@ 0x7f39e4a3461f clone at ??:0
I20260926 02:01:11.424156 30435 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=01eeb1c1fb824b99b09e00c9b9ab806e]
I20260926 02:01:11.424422 30435 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 33) took 2009 ms. Trace:
I20260926 02:01:11.424477 30435 rpcz_store.cc:276] 0926 02:01:09.414603 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414759 (+ 156us) service_pool.cc:224] Handling call
0926 02:01:09.978074 (+563315us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.978385 (+ 311us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.898539 (+920154us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.899114 (+ 575us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.899116 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.899234 (+ 118us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.424405 (+525171us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.899064 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.899073 (+ 9us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.899077 (+ 4us) write_op.cc:454] Released schema lock
0926 02:01:10.899081 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1290835,"Hive Metastore.run_cpu_time_us":356,"Hive Metastore.run_wall_time_us":153897,"mutex_wait_us":145,"child_traces":[["op",{}]]}
I20260926 02:01:11.485432 30434 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=6d4be6091284480ba5718a9d15ebabb6]
I20260926 02:01:11.485759 30434 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 36) took 2070 ms. Trace:
I20260926 02:01:11.485831 30434 rpcz_store.cc:276] 0926 02:01:09.414777 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414903 (+ 126us) service_pool.cc:224] Handling call
0926 02:01:09.978787 (+563884us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.979069 (+ 282us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:10.956498 (+977429us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:10.957000 (+ 502us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.957002 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:10.957124 (+ 122us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.485732 (+528608us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:10.956954 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:10.956962 (+ 8us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:10.956966 (+ 4us) write_op.cc:454] Released schema lock
0926 02:01:10.956969 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1386252,"Hive Metastore.run_cpu_time_us":411,"Hive Metastore.run_wall_time_us":119229,"mutex_wait_us":100,"child_traces":[["op",{}]]}
I20260926 02:01:11.524875 30432 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=c79138f00a124c90b674ad48ae7f2a19]
I20260926 02:01:11.525174 30432 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 34) took 2110 ms. Trace:
I20260926 02:01:11.525233 30432 rpcz_store.cc:276] 0926 02:01:09.414656 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414713 (+ 57us) service_pool.cc:224] Handling call
0926 02:01:09.979415 (+564702us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.979731 (+ 316us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:11.024512 (+1044781us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:11.025421 (+ 909us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:11.025424 (+ 3us) write_op.cc:454] Released schema lock
0926 02:01:11.025602 (+ 178us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.525154 (+499552us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:11.025352 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:11.025364 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:11.025370 (+ 6us) write_op.cc:454] Released schema lock
0926 02:01:11.025374 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1436393,"Hive Metastore.run_cpu_time_us":440,"Hive Metastore.run_wall_time_us":107302,"child_traces":[["op",{}]]}
I20260926 02:01:11.530476 31216 raft_consensus.cc:493] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:11.530625 31216 raft_consensus.cc:515] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.531205 31216 leader_election.cc:290] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:11.531833 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9043632256f4e8e9087287de5118010" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07" is_pre_election: true
I20260926 02:01:11.531984 30836 raft_consensus.cc:2468] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:11.532784 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9043632256f4e8e9087287de5118010" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" is_pre_election: true
I20260926 02:01:11.532934 30699 raft_consensus.cc:2468] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:11.533524 30908 leader_election.cc:304] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 08fbed94c9e248d0a89843d39b01ef07; no voters:
I20260926 02:01:11.533924 31216 raft_consensus.cc:2804] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:11.533986 31216 raft_consensus.cc:493] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:11.534031 31216 raft_consensus.cc:3060] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:11.535212 31216 raft_consensus.cc:515] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.535607 31216 leader_election.cc:290] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Requested vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:11.536458 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9043632256f4e8e9087287de5118010" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
I20260926 02:01:11.536582 30699 raft_consensus.cc:3060] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:11.537636 30699 raft_consensus.cc:2468] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:11.538098 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9043632256f4e8e9087287de5118010" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07"
I20260926 02:01:11.538239 30836 raft_consensus.cc:3060] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:11.539292 30836 raft_consensus.cc:2468] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:11.546794 30907 leader_election.cc:304] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:11.547322 31216 raft_consensus.cc:2804] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:11.547493 31216 raft_consensus.cc:697] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Becoming Leader. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Running, Role: LEADER
I20260926 02:01:11.547650 31216 consensus_queue.cc:237] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:11.549397 31018 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
W20260926 02:01:11.550216 30907 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): Couldn't send request to peer 92a655024b1b40dbbc5dc2cf5d6dd6fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 1: this message will repeat every 5th retry.
I20260926 02:01:11.564426 30431 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=640075642bc74658a5b1bf0ebf037db9]
I20260926 02:01:11.564723 30431 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:40936 (request call id 37) took 2149 ms. Trace:
I20260926 02:01:11.564779 30431 rpcz_store.cc:276] 0926 02:01:09.414831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 02:01:09.414911 (+ 80us) service_pool.cc:224] Handling call
0926 02:01:09.979679 (+564768us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 02:01:09.979984 (+ 305us) catalog_manager.cc:2186] Created new table and tablet info
0926 02:01:11.062113 (+1082129us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 02:01:11.062766 (+ 653us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:11.062768 (+ 2us) write_op.cc:454] Released schema lock
0926 02:01:11.062910 (+ 142us) catalog_manager.cc:2220] Rolling back HMS table creation
0926 02:01:11.564702 (+501792us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 02:01:11.062711 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d95dbe14730b406b8f4ffced8abe38f2 is not leader of this config: current role FOLLOWER)
0926 02:01:11.062719 (+ 8us) write_op.cc:489] Releasing partition, row and schema locks
0926 02:01:11.062724 (+ 5us) write_op.cc:454] Released schema lock
0926 02:01:11.062727 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1506266,"Hive Metastore.run_cpu_time_us":484,"Hive Metastore.run_wall_time_us":76982,"child_traces":[["op",{}]]}
W20260926 02:01:11.591262 30908 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Couldn't send request to peer 08fbed94c9e248d0a89843d39b01ef07. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 1: this message will repeat every 5th retry.
W20260926 02:01:11.631042 30769 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.631100 31025 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.631943 30634 connection.cc:573] server connection from 127.0.0.1:38896 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:11.632179 30905 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 02:01:11.874408 31284 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b285db155484443594c9385155358d4d)
I20260926 02:01:11.874516 31284 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:11.874979 31284 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:11.876349 30423 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:11.876469 30423 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:11.876539 30423 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:11.876657 31284 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 02:01:11.972836 30054 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu with pid 30405
I20260926 02:01:12.014132 31280 consensus_queue.cc:1048] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:12.020020 31216 consensus_queue.cc:1048] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:12.770388 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--webserver_interface=127.29.89.189
--webserver_port=44675
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:12.873965 31308 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:12.874207 31308 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:12.874276 31308 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:12.877882 31308 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:12.877974 31308 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:12.878013 31308 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:12.878075 31308 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:12.882375 31308 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.89.189
--webserver_port=44675
--never_fsync=true
--heap_profile_path=/tmp/kudu.31308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:12.883508 31308 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:12.884886 31308 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:12.890784 31313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:12.890827 31314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:12.891315 31316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:12.891516 31308 server_base.cc:1061] running on GCE node
I20260926 02:01:12.892057 31308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:12.892949 31308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:12.894127 31308 hybrid_clock.cc:648] HybridClock initialized: now 1790388072894117 us; error 39 us; skew 500 ppm
I20260926 02:01:12.895877 31308 webserver.cc:533] Webserver started at http://127.29.89.189:44675/ using document root <none> and password file <none>
I20260926 02:01:12.896430 31308 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:12.896526 31308 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:12.899806 31308 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:12.901726 31322 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:12.902727 31308 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:12.902871 31308 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:12.903270 31308 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:12.921777 31308 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:12.922485 31308 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:12.922698 31308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:12.930783 31308 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.189:33299
I20260926 02:01:12.930779 31374 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.189:33299 every 8 connection(s)
I20260926 02:01:12.931871 31308 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
I20260926 02:01:12.935796 31375 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:12.936203 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31308
I20260926 02:01:12.937808 31375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap starting.
I20260926 02:01:12.943339 31375 log.cc:826] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:12.953495 31375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=9 ignored=0} mutations{seen=5 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:12.953871 31375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap complete.
I20260926 02:01:12.956337 31375 raft_consensus.cc:359] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:12.956666 31375 raft_consensus.cc:740] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Initialized, Role: FOLLOWER
I20260926 02:01:12.957114 31375 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 4.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:12.957556 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:12.957665 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:12.958091 31375 sys_catalog.cc:565] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:12.962929 31393 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:12.963029 31393 catalog_manager.cc:1601] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: loading cluster ID for follower catalog manager: success
I20260926 02:01:12.962776 31308 master_runner.cc:187] Error getting master registration for 127.29.89.190:45707: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111),
I20260926 02:01:12.964851 31308 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:12.965504 31393 catalog_manager.cc:1623] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: acquiring CA information for follower catalog manager: success
I20260926 02:01:12.966702 31393 catalog_manager.cc:1651] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:13.004856 31020 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:13.006704 31339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:36285; Asking this server to re-register.
I20260926 02:01:13.007243 31020 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:13.008399 31339 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:13.027109 30748 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:13.028050 31339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:48149; Asking this server to re-register.
I20260926 02:01:13.028474 30748 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:13.029357 30884 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:13.029389 31339 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:13.030197 31339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:58647; Asking this server to re-register.
I20260926 02:01:13.030620 30884 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:13.031414 31339 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:13.553063 31408 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:13.553257 31408 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:13.554426 31408 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:13.554986 31323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:13.555835 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:13.555979 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:13.556035 31323 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:13.556236 31408 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 02:01:14.300139 31417 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:14.300390 31417 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:14.300987 31417 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:14.302449 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:14.302587 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:14.302668 31323 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:14.302924 31417 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 02:01:14.857766 31426 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:14.857909 31426 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:14.858479 31426 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:14.860236 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:14.860376 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:14.860417 31323 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:14.860571 31426 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 02:01:15.863137 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=36293
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:15.965622 31441 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:15.965818 31441 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:15.965858 31441 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:15.969238 31441 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:15.969290 31441 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:15.969308 31441 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:15.969326 31441 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:15.973116 31441 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=36293
--never_fsync=true
--heap_profile_path=/tmp/kudu.31441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:15.974072 31441 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:15.975219 31441 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:15.980409 31447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:15.980425 31449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:15.980412 31446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:15.980592 31441 server_base.cc:1061] running on GCE node
I20260926 02:01:15.981133 31441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:15.981968 31441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:15.983143 31441 hybrid_clock.cc:648] HybridClock initialized: now 1790388075983128 us; error 38 us; skew 500 ppm
I20260926 02:01:15.984900 31441 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:15.985446 31441 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:15.985527 31441 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:15.988823 31441 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:15.990686 31455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:15.991698 31441 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:15.991847 31441 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:15.992241 31441 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:16.012414 31441 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:16.013065 31441 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:16.013259 31441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:16.020632 31507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:16.020643 31441 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:16.021687 31441 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:16.025530 31508 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:16.027091 31508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:16.028977 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31441
I20260926 02:01:16.034472 31508 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:16.038834 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:16.039296 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:16.045964 31508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=9 ignored=0} mutations{seen=5 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:16.046271 31508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap complete.
I20260926 02:01:16.049178 31508 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.049469 31508 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:16.049861 31508 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 4.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.050333 31517 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:16.050482 31517 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:16.050895 31508 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:16.055577 31531 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:16.055661 31531 catalog_manager.cc:1601] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: success
I20260926 02:01:16.055402 31441 master_runner.cc:187] Error getting master registration for 127.29.89.190:45707: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111),
I20260926 02:01:16.057545 31441 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:16.062907 31531 catalog_manager.cc:1623] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: acquiring CA information for follower catalog manager: success
I20260926 02:01:16.064430 31531 catalog_manager.cc:1651] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:16.158387 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:16.158532 31018 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
I20260926 02:01:16.159885 31472 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:60159; Asking this server to re-register.
I20260926 02:01:16.160338 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:16.161448 31472 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:16.242285 31538 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:16.242463 31538 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.242935 31538 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:16.244498 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:16.246991 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d" is_pre_election: true
I20260926 02:01:16.247303 31482 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 4.
I20260926 02:01:16.247800 31323 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272
I20260926 02:01:16.247967 31538 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 02:01:16.248029 31538 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:16.248072 31538 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 4 FOLLOWER]: Advancing to term 5
I20260926 02:01:16.249750 31538 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.250099 31538 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:16.250648 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 5 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d"
I20260926 02:01:16.250787 31482 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 4 FOLLOWER]: Advancing to term 5
W20260926 02:01:16.251111 31323 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:16.252513 31482 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 5.
I20260926 02:01:16.252866 31323 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272
I20260926 02:01:16.253067 31538 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Leader election won for term 5
I20260926 02:01:16.253290 31538 raft_consensus.cc:697] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 LEADER]: Becoming Leader. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Running, Role: LEADER
I20260926 02:01:16.253633 31538 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 4.14, Last appended by leader: 14, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.255350 31542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 5 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:16.255460 31542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:16.255759 31544 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:16.257575 31544 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:16.258862 31544 tablet_loader.cc:96] loaded metadata for tablet e9043632256f4e8e9087287de5118010 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:16.259143 31544 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:16.259670 31544 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:16.259727 31544 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:16.260311 31544 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:16.260650 31544 catalog_manager.cc:6230] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Loaded TSK: 0
I20260926 02:01:16.261135 31544 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:16.261492 31544 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:16.261806 31544 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:16.357594 31335 master_service.cc:960] ReplaceTablet: received request to replace tablet e9043632256f4e8e9087287de5118010 from {username='slave'} at 127.0.0.1:58828
I20260926 02:01:16.357674 31337 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.357474 31339 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.357465 31338 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.358124 31336 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_aba582f363ee48b38f7b72fe1507952f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.358235 31330 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.358387 31332 master_service.cc:960] ReplaceTablet: received request to replace tablet e9043632256f4e8e9087287de5118010 from {username='slave'} at 127.0.0.1:58828
I20260926 02:01:16.358446 31331 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.358536 31334 master_service.cc:960] ReplaceTablet: received request to replace tablet e9043632256f4e8e9087287de5118010 from {username='slave'} at 127.0.0.1:58828
I20260926 02:01:16.358660 31333 master_service.cc:960] ReplaceTablet: received request to replace tablet e9043632256f4e8e9087287de5118010 from {username='slave'} at 127.0.0.1:58828
I20260926 02:01:16.362445 31482 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Refusing update from remote peer d95dbe14730b406b8f4ffced8abe38f2: Log matching property violated. Preceding OpId in replica: term: 4 index: 14. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20260926 02:01:16.363085 31542 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260926 02:01:16.363400 31323 consensus_peers.cc:597] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 -> Peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Couldn't send request to peer 995d22fd6e1b4cdeb3b8aa4c61b56272. Status: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 02:01:16.367136 31517 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 5 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:16.367262 31517 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:16.368737 31538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:16.368840 31538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:16.369938 31335 catalog_manager.cc:6133] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Sending DeleteTablet for 3 replicas of tablet e9043632256f4e8e9087287de5118010
I20260926 02:01:16.370293 31517 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:16.370440 31517 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
F20260926 02:01:16.371177 31392 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260926 02:01:16.371261 31335 catalog_manager.cc:6934] ReplaceTablet: tablet e9043632256f4e8e9087287de5118010 deleted and replaced by tablet 4b5d57f500e94587acbedbc1c6d8ab59
I20260926 02:01:16.372156 31335 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.374449 31332 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.374950 31334 master_service.cc:960] ReplaceTablet: received request to replace tablet e9043632256f4e8e9087287de5118010 from {username='slave'} at 127.0.0.1:58828
I20260926 02:01:16.375320 31334 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:16.375742 31333 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58828:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f927d606d5d google::LogMessage::Fail() at ??:0
I20260926 02:01:16.377604 30680 tablet_service.cc:1558] Processing DeleteTablet for tablet e9043632256f4e8e9087287de5118010 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b5d57f500e94587acbedbc1c6d8ab59) from {username='slave'} at 127.0.0.1:38904
I20260926 02:01:16.379516 30951 tablet_service.cc:1558] Processing DeleteTablet for tablet e9043632256f4e8e9087287de5118010 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b5d57f500e94587acbedbc1c6d8ab59) from {username='slave'} at 127.0.0.1:35374
I20260926 02:01:16.379798 31557 tablet_replica.cc:333] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: stopping tablet replica
I20260926 02:01:16.379941 31557 raft_consensus.cc:2243] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Raft consensus shutting down.
I20260926 02:01:16.380200 31557 raft_consensus.cc:2272] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:16.380679 31557 ts_tablet_manager.cc:1916] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:16.380939 30680 tablet_service.cc:1511] Processing CreateTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:16.381239 30680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b5d57f500e94587acbedbc1c6d8ab59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:16.383005 30815 tablet_service.cc:1511] Processing CreateTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:16.383142 31557 ts_tablet_manager.cc:1929] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:16.383213 31557 log.cc:1199] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/wals/e9043632256f4e8e9087287de5118010
I20260926 02:01:16.383293 30815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b5d57f500e94587acbedbc1c6d8ab59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:16.383484 31557 ts_tablet_manager.cc:1950] T e9043632256f4e8e9087287de5118010 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting consensus metadata
I20260926 02:01:16.383553 30816 tablet_service.cc:1558] Processing DeleteTablet for tablet e9043632256f4e8e9087287de5118010 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b5d57f500e94587acbedbc1c6d8ab59) from {username='slave'} at 127.0.0.1:33948
@ 0x7f927d60ab23 google::LogMessage::SendToLog() at ??:0
I20260926 02:01:16.384248 31324 catalog_manager.cc:5063] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): tablet e9043632256f4e8e9087287de5118010 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:16.385308 31560 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
I20260926 02:01:16.385479 30951 tablet_service.cc:1511] Processing CreateTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:16.385721 30951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b5d57f500e94587acbedbc1c6d8ab59. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f927d60675c google::LogMessage::Flush() at ??:0
I20260926 02:01:16.386382 31560 tablet_bootstrap.cc:654] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:16.387574 31560 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:16.387652 31560 ts_tablet_manager.cc:1403] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:16.387794 31562 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
@ 0x7f927d6069a1 google::LogMessage::~LogMessage() at ??:0
I20260926 02:01:16.388355 31560 raft_consensus.cc:359] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.388489 31560 raft_consensus.cc:385] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:16.388517 31560 raft_consensus.cc:740] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:16.388638 31560 consensus_queue.cc:260] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.388851 31558 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:16.388973 31560 ts_tablet_manager.cc:1434] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 02:01:16.389634 31559 tablet_replica.cc:333] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: stopping tablet replica
I20260926 02:01:16.389740 31559 raft_consensus.cc:2243] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:16.389855 31559 raft_consensus.cc:2272] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:16.390208 31559 ts_tablet_manager.cc:1916] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:16.390200 31472 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:38311; Asking this server to re-register.
I20260926 02:01:16.390739 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:16.391105 31562 tablet_bootstrap.cc:654] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:16.391701 31472 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:16.392310 31562 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:16.392308 31559 ts_tablet_manager.cc:1929] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:16.392378 31559 log.cc:1199] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/wals/e9043632256f4e8e9087287de5118010
I20260926 02:01:16.392396 31562 ts_tablet_manager.cc:1403] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 02:01:16.392643 31559 ts_tablet_manager.cc:1950] T e9043632256f4e8e9087287de5118010 P 08fbed94c9e248d0a89843d39b01ef07: Deleting consensus metadata
I20260926 02:01:16.392961 31558 tablet_bootstrap.cc:654] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:16.392954 31562 raft_consensus.cc:359] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.393087 31562 raft_consensus.cc:385] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:16.393132 31562 raft_consensus.cc:740] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:16.393296 31562 consensus_queue.cc:260] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.393596 31562 ts_tablet_manager.cc:1434] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:16.393877 31326 catalog_manager.cc:5063] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): tablet e9043632256f4e8e9087287de5118010 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
@ 0x7f9282bc7959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 02:01:16.395005 31555 tablet_replica.cc:333] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: stopping tablet replica
I20260926 02:01:16.395110 31555 raft_consensus.cc:2243] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:16.395239 31555 raft_consensus.cc:2272] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:16.395664 31555 ts_tablet_manager.cc:1916] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:16.397921 31558 tablet_bootstrap.cc:492] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
I20260926 02:01:16.397995 31558 ts_tablet_manager.cc:1403] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260926 02:01:16.398075 31555 ts_tablet_manager.cc:1929] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:16.398129 31555 log.cc:1199] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/wals/e9043632256f4e8e9087287de5118010
I20260926 02:01:16.398361 31555 ts_tablet_manager.cc:1950] T e9043632256f4e8e9087287de5118010 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting consensus metadata
I20260926 02:01:16.398398 31558 raft_consensus.cc:359] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.398483 31558 raft_consensus.cc:385] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:16.398504 31558 raft_consensus.cc:740] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:16.398633 31558 consensus_queue.cc:260] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.398990 31558 ts_tablet_manager.cc:1434] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:16.399395 31325 catalog_manager.cc:5063] TS 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): tablet e9043632256f4e8e9087287de5118010 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:16.400276 31472 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:53311; Asking this server to re-register.
I20260926 02:01:16.400831 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:16.401909 31472 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
@ 0x7f9282bc6870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f9282bc5fd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
@ 0x7f9282bcb6da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55bcb3aa8f7e std::function<>::operator()() at ??:0
@ 0x7f927eae2c6a kudu::Thread::SuperviseThread() at ??:0
@ 0x7f927f4616db start_thread at ??:0
@ 0x7f927c7bd61f clone at ??:0
W20260926 02:01:16.537689 30908 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Couldn't send request to peer 08fbed94c9e248d0a89843d39b01ef07. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 6: this message will repeat every 5th retry.
W20260926 02:01:16.572913 31026 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.189:33299)
W20260926 02:01:16.573087 31026 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.189:33299)
W20260926 02:01:16.575899 31026 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.189:33299)
W20260926 02:01:16.656625 30907 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): Couldn't send request to peer 92a655024b1b40dbbc5dc2cf5d6dd6fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 6: this message will repeat every 5th retry.
I20260926 02:01:16.668845 31281 raft_consensus.cc:493] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:16.668969 31281 raft_consensus.cc:515] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.669396 31281 leader_election.cc:290] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:16.669760 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b5d57f500e94587acbedbc1c6d8ab59" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" is_pre_election: true
I20260926 02:01:16.669934 30699 raft_consensus.cc:2468] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:16.669924 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b5d57f500e94587acbedbc1c6d8ab59" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07" is_pre_election: true
I20260926 02:01:16.670078 30836 raft_consensus.cc:2468] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:16.670331 30907 leader_election.cc:304] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:16.670519 31281 raft_consensus.cc:2804] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:16.670578 31281 raft_consensus.cc:493] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:16.670650 31281 raft_consensus.cc:3060] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:16.671769 31281 raft_consensus.cc:515] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.672152 31281 leader_election.cc:290] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Requested vote from peers 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981), 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:16.672437 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b5d57f500e94587acbedbc1c6d8ab59" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
I20260926 02:01:16.672560 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b5d57f500e94587acbedbc1c6d8ab59" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07"
I20260926 02:01:16.672597 30699 raft_consensus.cc:3060] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:16.672662 30836 raft_consensus.cc:3060] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:16.673569 30699 raft_consensus.cc:2468] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:16.673570 30836 raft_consensus.cc:2468] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:16.673885 30908 leader_election.cc:304] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 08fbed94c9e248d0a89843d39b01ef07; no voters:
I20260926 02:01:16.674058 31281 raft_consensus.cc:2804] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:16.674152 31281 raft_consensus.cc:697] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Becoming Leader. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Running, Role: LEADER
I20260926 02:01:16.674352 31281 consensus_queue.cc:237] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:16.879505 31581 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d95dbe14730b406b8f4ffced8abe38f2)
I20260926 02:01:16.879657 31581 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:16.880892 31581 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:16.881413 31456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:16.882205 31457 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:16.882367 31456 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:16.882453 31456 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:16.882681 31581 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:17.192967 31281 consensus_queue.cc:1048] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:17.197034 31281 consensus_queue.cc:1048] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:17.421314 31595 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d95dbe14730b406b8f4ffced8abe38f2)
I20260926 02:01:17.421487 31595 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:17.422034 31595 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:17.423264 31456 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:17.423418 31457 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:17.423497 31457 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:17.423666 31595 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:18.105289 30054 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu with pid 31441
I20260926 02:01:18.682927 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=36293
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:18.787923 31610 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:18.788115 31610 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:18.788153 31610 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:18.791435 31610 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:18.791486 31610 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:18.791507 31610 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:18.791558 31610 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:18.795395 31610 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=36293
--never_fsync=true
--heap_profile_path=/tmp/kudu.31610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:18.796424 31610 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:18.797533 31610 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:18.802953 31620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:18.802940 31622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:18.803036 31619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:18.803169 31610 server_base.cc:1061] running on GCE node
I20260926 02:01:18.803751 31610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:18.804592 31610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:18.805779 31610 hybrid_clock.cc:648] HybridClock initialized: now 1790388078805751 us; error 38 us; skew 500 ppm
I20260926 02:01:18.807956 31610 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:18.808566 31610 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:18.808625 31610 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:18.812083 31610 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:18.814074 31628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:18.815089 31610 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:18.815208 31610 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:18.815598 31610 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:18.840137 31610 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:18.840795 31610 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:18.840991 31610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:18.848187 31680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:18.848197 31610 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:18.849365 31610 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:18.853156 31681 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:18.854724 31681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:18.859469 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31610
I20260926 02:01:18.860913 31681 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:18.872592 31681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=10 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:18.872911 31681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap complete.
I20260926 02:01:18.875823 31681 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:18.876151 31681 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:18.876627 31681 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:18.877108 31685 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:18.877238 31685 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:18.877893 31681 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:18.883371 31699 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:18.883445 31699 catalog_manager.cc:1601] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: success
I20260926 02:01:18.882524 31610 master_runner.cc:187] Error getting master registration for 127.29.89.190:45707: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111),
I20260926 02:01:18.885030 31610 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:18.885434 31699 catalog_manager.cc:1623] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: acquiring CA information for follower catalog manager: success
I20260926 02:01:18.886684 31699 catalog_manager.cc:1651] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:19.149427 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:19.151325 31645 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:55193; Asking this server to re-register.
I20260926 02:01:19.151997 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:19.153254 31645 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:19.206636 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:19.207520 31645 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:53985; Asking this server to re-register.
I20260926 02:01:19.207988 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:19.208554 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:19.208952 31645 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:19.209462 31644 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:57591; Asking this server to re-register.
I20260926 02:01:19.209833 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:19.210655 31644 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:19.434047 31710 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:19.434262 31710 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:19.435581 31710 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:19.436275 31629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:19.437224 31629 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:19.437350 31630 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:19.437439 31630 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:19.437633 31710 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:19.993291 31716 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:19.993422 31716 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:19.994058 31716 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:19.995594 31629 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:19.995703 31630 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:19.995775 31630 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:19.995899 31716 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:20.744107 31730 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:20.744252 31730 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:20.744668 31730 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:20.746155 31629 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:20.746291 31630 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:20.746397 31630 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:20.746574 31730 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:21.115173 30054 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu with pid 31610
W20260926 02:01:21.122519 30905 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:21.567289 30907 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): Couldn't send request to peer 92a655024b1b40dbbc5dc2cf5d6dd6fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 11: this message will repeat every 5th retry.
I20260926 02:01:21.648278 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=36293
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:21.756045 31742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:21.756256 31742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:21.756295 31742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:21.759653 31742 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:21.759717 31742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:21.759738 31742 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:21.759755 31742 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:21.763664 31742 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=36293
--never_fsync=true
--heap_profile_path=/tmp/kudu.31742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:21.764653 31742 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:21.765722 31742 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:21.770996 31750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:21.771008 31748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:21.771008 31747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:21.771598 31742 server_base.cc:1061] running on GCE node
I20260926 02:01:21.772011 31742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:21.772847 31742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:21.774024 31742 hybrid_clock.cc:648] HybridClock initialized: now 1790388081774005 us; error 29 us; skew 500 ppm
I20260926 02:01:21.775931 31742 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:21.776505 31742 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:21.776563 31742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:21.780194 31742 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:21.782127 31756 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:21.783118 31742 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 02:01:21.783262 31742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:21.783646 31742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:21.801504 31742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:21.802140 31742 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:21.802347 31742 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260926 02:01:21.809388 30908 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Couldn't send request to peer 08fbed94c9e248d0a89843d39b01ef07. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 11: this message will repeat every 5th retry.
I20260926 02:01:21.810019 31742 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:21.810043 31809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:21.811411 31742 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:21.814438 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31742
I20260926 02:01:21.815429 31810 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:21.817320 31810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:21.823946 31810 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:21.834206 31810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=10 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:21.834604 31810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap complete.
I20260926 02:01:21.837097 31810 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:21.837443 31810 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:21.837904 31810 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:21.838434 31814 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:21.838541 31814 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:21.838967 31810 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:21.844403 31828 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:21.844475 31828 catalog_manager.cc:1601] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: success
I20260926 02:01:21.843473 31742 master_runner.cc:187] Error getting master registration for 127.29.89.190:45707: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111),
I20260926 02:01:21.845989 31742 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:21.846750 31828 catalog_manager.cc:1623] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: acquiring CA information for follower catalog manager: success
I20260926 02:01:21.847918 31828 catalog_manager.cc:1651] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:22.103245 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:22.105608 31773 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:55371; Asking this server to re-register.
I20260926 02:01:22.106209 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:22.107594 31773 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:22.219178 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:22.219981 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:22.220093 31772 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:45935; Asking this server to re-register.
I20260926 02:01:22.220472 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:22.220727 31772 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:33003; Asking this server to re-register.
I20260926 02:01:22.221095 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:22.221290 31772 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:22.221872 31772 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:22.400875 31836 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:22.401055 31836 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:22.402822 31836 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:22.403232 31758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:22.404057 31757 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:22.404204 31758 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:22.404292 31758 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:22.404531 31836 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:22.954146 31841 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:22.954381 31841 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:22.955005 31841 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:22.956571 31758 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:22.956669 31757 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:22.956739 31757 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:22.956885 31841 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:23.848115 31854 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:23.848249 31854 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:23.848810 31854 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:23.850096 31758 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:23.850188 31757 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:23.850286 31757 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:23.850478 31854 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:24.735172 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--webserver_interface=127.29.89.190
--webserver_port=41125
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:24.837337 31867 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:24.837585 31867 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:24.837666 31867 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:24.841156 31867 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:24.841243 31867 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:24.841279 31867 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:24.841337 31867 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:24.845356 31867 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.89.190
--webserver_port=41125
--never_fsync=true
--heap_profile_path=/tmp/kudu.31867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:24.846469 31867 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:24.847579 31867 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:24.853193 31875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:24.853070 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:24.853070 31873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:24.853229 31867 server_base.cc:1061] running on GCE node
I20260926 02:01:24.853896 31867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:24.854843 31867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:24.856034 31867 hybrid_clock.cc:648] HybridClock initialized: now 1790388084856012 us; error 45 us; skew 500 ppm
I20260926 02:01:24.857743 31867 webserver.cc:533] Webserver started at http://127.29.89.190:41125/ using document root <none> and password file <none>
I20260926 02:01:24.858376 31867 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:24.858475 31867 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:24.861613 31867 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:24.863556 31881 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:24.864481 31867 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 02:01:24.864612 31867 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:24.865000 31867 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:24.880568 31867 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:24.881181 31867 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:24.881373 31867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:24.889334 31867 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.190:45707
I20260926 02:01:24.889361 31933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.190:45707 every 8 connection(s)
I20260926 02:01:24.890439 31867 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
I20260926 02:01:24.890906 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 31867
I20260926 02:01:24.894585 31934 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:24.896221 31934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap starting.
I20260926 02:01:24.902212 31934 log.cc:826] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:24.911023 31934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=8 ignored=0} mutations{seen=3 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:24.911386 31934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap complete.
I20260926 02:01:24.914270 31934 raft_consensus.cc:359] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:24.914628 31934 raft_consensus.cc:740] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 995d22fd6e1b4cdeb3b8aa4c61b56272, State: Initialized, Role: FOLLOWER
I20260926 02:01:24.915069 31934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 3.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:24.915509 31939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:24.915632 31939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:24.916167 31934 sys_catalog.cc:565] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:24.921657 31019 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:24.921790 31019 heartbeater.cc:499] Master 127.29.89.190:45707 was elected leader, sending a full tablet report...
I20260926 02:01:24.924074 31898 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:37597; Asking this server to re-register.
I20260926 02:01:24.924794 31019 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:24.925949 31897 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:24.926849 31954 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:24.926936 31954 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: loading cluster ID for follower catalog manager: success
I20260926 02:01:24.926611 31867 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:24.928618 31867 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:24.929196 31954 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: acquiring CA information for follower catalog manager: success
I20260926 02:01:24.930262 31954 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:25.137511 31958 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:25.137668 31958 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:25.138135 31958 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:25.139353 31758 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:25.142259 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:25.142637 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 3.
I20260926 02:01:25.143164 31757 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:25.143359 31958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260926 02:01:25.143447 31958 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:25.143524 31958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 5 FOLLOWER]: Advancing to term 6
I20260926 02:01:25.145253 31958 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:25.145634 31958 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:25.145985 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:25.146127 31908 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 3 FOLLOWER]: Advancing to term 6
W20260926 02:01:25.146628 31758 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:25.147825 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 6.
I20260926 02:01:25.148182 31757 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:25.148463 31958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Leader election won for term 6
I20260926 02:01:25.148713 31958 raft_consensus.cc:697] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 LEADER]: Becoming Leader. State: Replica: b285db155484443594c9385155358d4d, State: Running, Role: LEADER
I20260926 02:01:25.149013 31958 consensus_queue.cc:237] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:25.150710 31962 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 6 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:25.150797 31962 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:25.151101 31964 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:25.152482 31964 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:25.153349 31964 tablet_loader.cc:96] loaded metadata for tablet 4b5d57f500e94587acbedbc1c6d8ab59 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:25.153762 31964 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:25.154099 31964 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:25.154142 31964 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:25.154696 31964 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:25.155045 31964 catalog_manager.cc:6230] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Loaded TSK: 0
I20260926 02:01:25.155531 31964 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:25.155896 31964 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:25.156215 31964 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:25.166483 31773 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41556:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
I20260926 02:01:25.166682 31772 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.166790 31771 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167124 31770 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167217 31767 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167137 31769 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167372 31768 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167603 31764 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167567 31766 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:25.167693 31765 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41556:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 02:01:25.172786 31827 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f0040b28d5d google::LogMessage::Fail() at ??:0
@ 0x7f0040b2cb23 google::LogMessage::SendToLog() at ??:0
@ 0x7f0040b2875c google::LogMessage::Flush() at ??:0
@ 0x7f0040b289a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f00460e9959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f00460e8870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f00460e7fd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
@ 0x7f00460ed6da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x558e26b5ff7e std::function<>::operator()() at ??:0
@ 0x7f0042004c6a kudu::Thread::SuperviseThread() at ??:0
@ 0x7f00429836db start_thread at ??:0
@ 0x7f003fcdf61f clone at ??:0
I20260926 02:01:25.221252 30883 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:25.221302 30747 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:25.222201 31897 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:34225; Asking this server to re-register.
I20260926 02:01:25.222271 31898 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:46263; Asking this server to re-register.
I20260926 02:01:25.222766 30747 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:25.222779 30883 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:25.223606 31898 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:25.223716 31897 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
W20260926 02:01:25.348938 30633 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:25.348953 30769 connection.cc:573] client connection to 127.29.89.188:46477 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:25.349658 31025 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.188:46477)
W20260926 02:01:25.350003 31025 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.188:46477)
W20260926 02:01:25.350327 31025 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.188:46477)
I20260926 02:01:25.810878 31986 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:25.811017 31986 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:25.812333 31986 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:25.812978 31882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:25.813752 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:25.813875 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:25.813970 31883 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:25.814237 31986 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 02:01:26.361603 31991 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:26.361714 31991 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:26.362211 31991 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:26.363365 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:26.363512 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:26.363597 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:26.363746 31991 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 02:01:26.502717 30907 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): Couldn't send request to peer 92a655024b1b40dbbc5dc2cf5d6dd6fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 16: this message will repeat every 5th retry.
W20260926 02:01:26.900781 30908 consensus_peers.cc:597] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe -> Peer 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): Couldn't send request to peer 08fbed94c9e248d0a89843d39b01ef07. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee779b26708f4c5e95b2a007ff0ccb5e. This is attempt 16: this message will repeat every 5th retry.
I20260926 02:01:27.044601 31997 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:27.044736 31997 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:27.045151 31997 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:27.046480 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:27.046787 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:27.046875 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:27.047067 31997 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 02:01:28.077840 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--webserver_interface=127.29.89.189
--webserver_port=44675
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:28.188714 32004 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:28.188917 32004 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:28.188957 32004 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:28.192763 32004 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:28.192823 32004 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:28.192844 32004 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:28.192862 32004 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:28.196764 32004 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.89.189
--webserver_port=44675
--never_fsync=true
--heap_profile_path=/tmp/kudu.32004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:28.197733 32004 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:28.198773 32004 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:28.203943 32010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:28.203949 32012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:28.203949 32009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:28.204209 32004 server_base.cc:1061] running on GCE node
I20260926 02:01:28.204726 32004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:28.205614 32004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:28.206805 32004 hybrid_clock.cc:648] HybridClock initialized: now 1790388088206783 us; error 45 us; skew 500 ppm
I20260926 02:01:28.208495 32004 webserver.cc:533] Webserver started at http://127.29.89.189:44675/ using document root <none> and password file <none>
I20260926 02:01:28.209116 32004 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:28.209199 32004 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:28.212644 32004 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 02:01:28.214612 32018 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:28.215605 32004 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 02:01:28.215736 32004 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:28.216117 32004 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:28.228883 32004 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:28.229476 32004 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:28.229671 32004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:28.237776 32070 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.189:33299 every 8 connection(s)
I20260926 02:01:28.237785 32004 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.189:33299
I20260926 02:01:28.238866 32004 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
I20260926 02:01:28.242578 32071 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:28.244050 32071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap starting.
I20260926 02:01:28.244324 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 32004
I20260926 02:01:28.250923 32071 log.cc:826] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:28.261278 32071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=10 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:28.261636 32071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap complete.
I20260926 02:01:28.264096 32071 raft_consensus.cc:359] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:28.264420 32071 raft_consensus.cc:740] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Initialized, Role: FOLLOWER
I20260926 02:01:28.264873 32071 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:28.265303 32075 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:28.265404 32075 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:28.265837 32071 sys_catalog.cc:565] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:28.271070 32089 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:28.271123 32089 catalog_manager.cc:1601] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: loading cluster ID for follower catalog manager: success
I20260926 02:01:28.273036 32089 catalog_manager.cc:1623] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: acquiring CA information for follower catalog manager: success
I20260926 02:01:28.274385 32089 catalog_manager.cc:1651] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:28.287786 32004 master_runner.cc:187] Error getting master registration for 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111),
I20260926 02:01:28.289273 32004 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:28.409591 31020 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:28.411686 32035 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:37173; Asking this server to re-register.
I20260926 02:01:28.412233 31020 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:28.413321 32035 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:28.473778 32094 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:28.473894 32094 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:28.474412 32094 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:28.475669 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:28.478161 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 7 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: false dest_uuid: "d95dbe14730b406b8f4ffced8abe38f2" is_pre_election: true
I20260926 02:01:28.478505 32045 raft_consensus.cc:2410] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 995d22fd6e1b4cdeb3b8aa4c61b56272 for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 11.
I20260926 02:01:28.478965 31883 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:28.479135 32094 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 02:01:28.727316 30884 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:28.727593 30748 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:28.728335 32035 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:41181; Asking this server to re-register.
I20260926 02:01:28.728650 32034 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:51277; Asking this server to re-register.
I20260926 02:01:28.728719 30884 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:28.729067 30748 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:28.729535 32034 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:28.729964 32035 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:28.756486 32075 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:28.756695 32075 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:28.757934 32075 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:28.758512 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:28.758731 31908 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d95dbe14730b406b8f4ffced8abe38f2 in current term 6: Already voted for candidate b285db155484443594c9385155358d4d in this term.
W20260926 02:01:28.758764 32019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:28.759572 32019 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:28.759675 32019 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d95dbe14730b406b8f4ffced8abe38f2; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d
I20260926 02:01:28.759848 32075 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 5 FOLLOWER]: Advancing to term 6
I20260926 02:01:28.761587 32075 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 02:01:29.429055 32104 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:29.429250 32104 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:29.430119 32104 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:29.430650 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 7 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:29.430840 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 6.
I20260926 02:01:29.431187 32019 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:29.431420 32104 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260926 02:01:29.431505 32104 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:29.431568 32104 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 6 FOLLOWER]: Advancing to term 7
W20260926 02:01:29.431710 32019 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:29.432605 32104 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:29.432968 32104 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:29.433396 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 7 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:29.433533 31908 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 6 FOLLOWER]: Advancing to term 7
I20260926 02:01:29.434469 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 7.
W20260926 02:01:29.434486 32019 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:29.434970 32019 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters: b285db155484443594c9385155358d4d
I20260926 02:01:29.435249 32104 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Leader election won for term 7
I20260926 02:01:29.435431 32104 raft_consensus.cc:697] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 LEADER]: Becoming Leader. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Running, Role: LEADER
I20260926 02:01:29.435748 32104 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:29.437357 32107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.437469 32107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:29.437966 32109 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:29.439535 32109 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:29.440472 32109 tablet_loader.cc:96] loaded metadata for tablet 4b5d57f500e94587acbedbc1c6d8ab59 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:29.440956 32109 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:29.441381 32109 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:29.441426 32109 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:29.441959 32109 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:29.442440 32109 catalog_manager.cc:6230] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Loaded TSK: 0
I20260926 02:01:29.442994 32109 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:29.443395 32109 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:29.443784 32109 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:29.478564 30951 tablet_service.cc:1558] Processing DeleteTablet for tablet ee779b26708f4c5e95b2a007ff0ccb5e with delete_type TABLET_DATA_DELETED (replaced by tablet d63b60f303f74dca95eebb234fcb6316) from {username='slave'} at 127.0.0.1:57852
I20260926 02:01:29.478894 32112 tablet_replica.cc:333] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: stopping tablet replica
I20260926 02:01:29.479055 32112 raft_consensus.cc:2243] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Raft consensus shutting down.
I20260926 02:01:29.479333 32112 pending_rounds.cc:70] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Trying to abort 1 pending ops.
I20260926 02:01:29.479537 32112 pending_rounds.cc:77] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333429538703941632 op_type: NO_OP noop_request { }
I20260926 02:01:29.479657 32112 raft_consensus.cc:2889] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 02:01:29.479730 32112 raft_consensus.cc:2272] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:29.480104 32112 ts_tablet_manager.cc:1916] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:29.482247 32112 ts_tablet_manager.cc:1929] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:29.482375 32112 log.cc:1199] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/wals/ee779b26708f4c5e95b2a007ff0ccb5e
I20260926 02:01:29.482764 32112 ts_tablet_manager.cc:1950] T ee779b26708f4c5e95b2a007ff0ccb5e P 02a024f3eff64b34b0fa7e46d6274abe: Deleting consensus metadata
I20260926 02:01:29.483450 32020 catalog_manager.cc:5063] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): tablet ee779b26708f4c5e95b2a007ff0ccb5e (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:29.488672 32035 master_service.cc:960] ReplaceTablet: received request to replace tablet 4b5d57f500e94587acbedbc1c6d8ab59 from {username='slave'} at 127.0.0.1:33626
I20260926 02:01:29.488720 32034 master_service.cc:960] ReplaceTablet: received request to replace tablet 4b5d57f500e94587acbedbc1c6d8ab59 from {username='slave'} at 127.0.0.1:33626
I20260926 02:01:29.488893 32032 master_service.cc:960] ReplaceTablet: received request to replace tablet 4b5d57f500e94587acbedbc1c6d8ab59 from {username='slave'} at 127.0.0.1:33626
I20260926 02:01:29.489059 32029 master_service.cc:960] ReplaceTablet: received request to replace tablet 4b5d57f500e94587acbedbc1c6d8ab59 from {username='slave'} at 127.0.0.1:33626
I20260926 02:01:29.489193 32033 master_service.cc:960] ReplaceTablet: received request to replace tablet 4b5d57f500e94587acbedbc1c6d8ab59 from {username='slave'} at 127.0.0.1:33626
I20260926 02:01:29.489369 32031 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_aba582f363ee48b38f7b72fe1507952f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.489429 32030 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.489697 32028 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.489871 32026 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.489941 32027 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.493398 31908 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Refusing update from remote peer d95dbe14730b406b8f4ffced8abe38f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 11. Preceding OpId from leader: term: 7 index: 19. (index mismatch)
W20260926 02:01:29.493741 32019 consensus_peers.cc:597] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 -> Peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Couldn't send request to peer b285db155484443594c9385155358d4d. Status: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 02:01:29.494002 32107 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 11, Time since last communication: 0.000s
F20260926 02:01:29.497394 32088 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260926 02:01:29.499920 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.500065 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:29.500173 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.500249 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:29.500326 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.500396 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:29.500715 32115 mvcc.cc:204] Tried to move back new op lower bound from 7333429560827342848 to 7333429540179517440. Current Snapshot: MvccSnapshot[applied={T|T < 7333429560827342848 or (T in {7333429560827342848})}]
@ 0x7fe8a0cf5d5d google::LogMessage::Fail() at ??:0
I20260926 02:01:29.501899 32104 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.501994 32104 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:29.502384 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:29.502539 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:29.503106 32113 mvcc.cc:204] Tried to move back new op lower bound from 7333429614558310400 to 7333429614331195392. Current Snapshot: MvccSnapshot[applied={T|T < 7333429614558310400 or (T in {7333429614558310400})}]
@ 0x7fe8a0cf9b23 google::LogMessage::SendToLog() at ??:0
I20260926 02:01:29.503693 32032 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.504004 32034 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.504156 32035 catalog_manager.cc:6133] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Sending DeleteTablet for 3 replicas of tablet 4b5d57f500e94587acbedbc1c6d8ab59
I20260926 02:01:29.504146 32029 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.503711 32033 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33626:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
I20260926 02:01:29.505259 32035 catalog_manager.cc:6934] ReplaceTablet: tablet 4b5d57f500e94587acbedbc1c6d8ab59 deleted and replaced by tablet 134188f591ff45bbb46f452b4b018662
I20260926 02:01:29.505493 32035 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:29.505586 30951 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 with delete_type TABLET_DATA_DELETED (replaced by tablet 134188f591ff45bbb46f452b4b018662) from {username='slave'} at 127.0.0.1:57852
@ 0x7fe8a0cf575c google::LogMessage::Flush() at ??:0
I20260926 02:01:29.505765 32112 tablet_replica.cc:333] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: stopping tablet replica
I20260926 02:01:29.505887 32112 raft_consensus.cc:2243] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Raft consensus shutting down.
I20260926 02:01:29.506083 32112 raft_consensus.cc:2272] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:29.506538 32112 ts_tablet_manager.cc:1916] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fe8a0cf59a1 google::LogMessage::~LogMessage() at ??:0
I20260926 02:01:29.508975 32112 ts_tablet_manager.cc:1929] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:29.509050 32112 log.cc:1199] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/wals/4b5d57f500e94587acbedbc1c6d8ab59
I20260926 02:01:29.509310 32112 ts_tablet_manager.cc:1950] T 4b5d57f500e94587acbedbc1c6d8ab59 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting consensus metadata
I20260926 02:01:29.510634 32020 catalog_manager.cc:5063] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): tablet 4b5d57f500e94587acbedbc1c6d8ab59 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
@ 0x7fe8a62b6959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 02:01:29.512953 30951 tablet_service.cc:1511] Processing CreateTablet for tablet 134188f591ff45bbb46f452b4b018662 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:29.513190 30951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134188f591ff45bbb46f452b4b018662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:29.516000 30680 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 with delete_type TABLET_DATA_DELETED (replaced by tablet 134188f591ff45bbb46f452b4b018662) from {username='slave'} at 127.0.0.1:45734
I20260926 02:01:29.516319 32128 tablet_replica.cc:333] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: stopping tablet replica
I20260926 02:01:29.516278 30679 tablet_service.cc:1511] Processing CreateTablet for tablet 134188f591ff45bbb46f452b4b018662 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:29.516422 32128 raft_consensus.cc:2243] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:29.516520 30679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134188f591ff45bbb46f452b4b018662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:29.516549 32128 raft_consensus.cc:2272] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:29.517863 30815 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b5d57f500e94587acbedbc1c6d8ab59 with delete_type TABLET_DATA_DELETED (replaced by tablet 134188f591ff45bbb46f452b4b018662) from {username='slave'} at 127.0.0.1:34032
I20260926 02:01:29.517982 32128 ts_tablet_manager.cc:1916] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:29.518078 30816 tablet_service.cc:1511] Processing CreateTablet for tablet 134188f591ff45bbb46f452b4b018662 (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:29.518409 30816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134188f591ff45bbb46f452b4b018662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:29.518762 32130 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:29.519788 32130 tablet_bootstrap.cc:654] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:29.520720 32128 ts_tablet_manager.cc:1929] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:29.520747 32132 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
I20260926 02:01:29.520795 32128 log.cc:1199] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/wals/4b5d57f500e94587acbedbc1c6d8ab59
I20260926 02:01:29.521123 32128 ts_tablet_manager.cc:1950] T 4b5d57f500e94587acbedbc1c6d8ab59 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting consensus metadata
@ 0x7fe8a62b5870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 02:01:29.521757 32132 tablet_bootstrap.cc:654] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:29.522148 32021 catalog_manager.cc:5063] TS 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): tablet 4b5d57f500e94587acbedbc1c6d8ab59 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:29.522867 32132 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:29.522933 32132 ts_tablet_manager.cc:1403] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:29.523307 32132 raft_consensus.cc:359] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.523387 32132 raft_consensus.cc:385] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:29.523408 32132 raft_consensus.cc:740] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:29.523581 32132 consensus_queue.cc:260] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.523885 32132 ts_tablet_manager.cc:1434] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
@ 0x7fe8a62b4fd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
@ 0x7fe8a62ba6da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
I20260926 02:01:29.530061 32129 tablet_replica.cc:333] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: stopping tablet replica
@ 0x556ae72d0f7e std::function<>::operator()() at ??:0
I20260926 02:01:29.530225 32129 raft_consensus.cc:2243] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:29.530447 32129 raft_consensus.cc:2272] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:29.530692 32127 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
I20260926 02:01:29.530897 32129 ts_tablet_manager.cc:1916] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:29.532099 32127 tablet_bootstrap.cc:654] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:29.533473 32129 ts_tablet_manager.cc:1929] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:29.533552 32129 log.cc:1199] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/wals/4b5d57f500e94587acbedbc1c6d8ab59
@ 0x7fe8a21d1c6a kudu::Thread::SuperviseThread() at ??:0
I20260926 02:01:29.533608 32130 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
I20260926 02:01:29.533710 32130 ts_tablet_manager.cc:1403] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260926 02:01:29.533834 32129 ts_tablet_manager.cc:1950] T 4b5d57f500e94587acbedbc1c6d8ab59 P 08fbed94c9e248d0a89843d39b01ef07: Deleting consensus metadata
I20260926 02:01:29.533923 32127 tablet_bootstrap.cc:492] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:29.534044 32127 ts_tablet_manager.cc:1403] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 02:01:29.534372 32130 raft_consensus.cc:359] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.534523 32130 raft_consensus.cc:385] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:29.534572 32130 raft_consensus.cc:740] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:29.534657 32127 raft_consensus.cc:359] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.534781 32127 raft_consensus.cc:385] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:29.534734 32130 consensus_queue.cc:260] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.534822 32127 raft_consensus.cc:740] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:29.535177 32130 ts_tablet_manager.cc:1434] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:29.535348 32127 consensus_queue.cc:260] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.536134 31020 heartbeater.cc:499] Master 127.29.89.189:33299 was elected leader, sending a full tablet report...
I20260926 02:01:29.536661 32127 ts_tablet_manager.cc:1434] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 02:01:29.536906 32022 catalog_manager.cc:5063] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): tablet 4b5d57f500e94587acbedbc1c6d8ab59 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
@ 0x7fe8a2b506db start_thread at ??:0
@ 0x7fe89feac61f clone at ??:0
W20260926 02:01:29.696765 31026 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.189:33299)
W20260926 02:01:29.697113 31026 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.189:33299)
W20260926 02:01:29.697762 32102 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 02:01:29.696977 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.29.89.189:33299 (local address 127.29.89.130:57713)
0926 02:01:29.697183 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 02:01:29.697200 (+ 17us) client_negotiation.cc:175] Beginning negotiation
0926 02:01:29.697299 (+ 99us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 02:01:29.697633 (+ 334us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":119}
W20260926 02:01:29.699162 32103 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 02:01:29.696977 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.29.89.189:33299 (local address 127.29.89.129:47461)
0926 02:01:29.697183 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 02:01:29.697200 (+ 17us) client_negotiation.cc:175] Beginning negotiation
0926 02:01:29.697299 (+ 99us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 02:01:29.698539 (+ 1240us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":120}
W20260926 02:01:29.705734 31026 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.189:33299)
I20260926 02:01:29.736119 32138 raft_consensus.cc:493] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:29.736275 32138 raft_consensus.cc:515] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.737349 32138 leader_election.cc:290] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:29.740754 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134188f591ff45bbb46f452b4b018662" candidate_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07" is_pre_election: true
I20260926 02:01:29.740929 30836 raft_consensus.cc:2468] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92a655024b1b40dbbc5dc2cf5d6dd6fb in term 0.
I20260926 02:01:29.741061 30972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134188f591ff45bbb46f452b4b018662" candidate_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02a024f3eff64b34b0fa7e46d6274abe" is_pre_election: true
I20260926 02:01:29.741268 30972 raft_consensus.cc:2468] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92a655024b1b40dbbc5dc2cf5d6dd6fb in term 0.
I20260926 02:01:29.741338 30636 leader_election.cc:304] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08fbed94c9e248d0a89843d39b01ef07, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:29.741626 32138 raft_consensus.cc:2804] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:29.741695 32138 raft_consensus.cc:493] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:29.741747 32138 raft_consensus.cc:3060] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:29.742708 32138 raft_consensus.cc:515] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.743074 32138 leader_election.cc:290] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [CANDIDATE]: Term 1 election: Requested vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:29.743454 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134188f591ff45bbb46f452b4b018662" candidate_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07"
I20260926 02:01:29.743480 30972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134188f591ff45bbb46f452b4b018662" candidate_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02a024f3eff64b34b0fa7e46d6274abe"
I20260926 02:01:29.743582 30972 raft_consensus.cc:3060] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:29.743582 30836 raft_consensus.cc:3060] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:29.744513 30836 raft_consensus.cc:2468] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92a655024b1b40dbbc5dc2cf5d6dd6fb in term 1.
I20260926 02:01:29.744563 30972 raft_consensus.cc:2468] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92a655024b1b40dbbc5dc2cf5d6dd6fb in term 1.
I20260926 02:01:29.744839 30636 leader_election.cc:304] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08fbed94c9e248d0a89843d39b01ef07, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:29.745045 32138 raft_consensus.cc:2804] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:29.745258 32138 raft_consensus.cc:697] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 LEADER]: Becoming Leader. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Running, Role: LEADER
I20260926 02:01:29.745834 32138 consensus_queue.cc:237] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:29.930481 31867 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:29.930606 31867 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:30.012128 32152 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d95dbe14730b406b8f4ffced8abe38f2)
I20260926 02:01:30.012305 32152 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:30.012789 32152 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:30.014142 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:30.014254 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:30.014338 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:30.014495 32152 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 02:01:30.297559 32157 consensus_queue.cc:1048] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:30.302101 32163 consensus_queue.cc:1048] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:30.569234 32170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d95dbe14730b406b8f4ffced8abe38f2)
I20260926 02:01:30.569406 32170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:30.569924 32170 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:30.571103 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:30.571206 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:30.571283 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:30.571408 32170 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 02:01:31.182046 32178 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d95dbe14730b406b8f4ffced8abe38f2)
I20260926 02:01:31.182240 32178 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:31.182793 32178 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:31.183324 31883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260926 02:01:31.184067 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:31.184303 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:31.184357 31883 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:31.184487 32178 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 02:01:31.430601 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--webserver_interface=127.29.89.189
--webserver_port=44675
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:31.534967 32187 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:31.535166 32187 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:31.535241 32187 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:31.538565 32187 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:31.538615 32187 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:31.538635 32187 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:31.538652 32187 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:31.542502 32187 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.189:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.89.189
--webserver_port=44675
--never_fsync=true
--heap_profile_path=/tmp/kudu.32187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:31.543422 32187 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:31.544453 32187 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:31.549602 32193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:31.549676 32192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:31.549611 32195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:31.549971 32187 server_base.cc:1061] running on GCE node
I20260926 02:01:31.550499 32187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:31.551317 32187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:31.552475 32187 hybrid_clock.cc:648] HybridClock initialized: now 1790388091552461 us; error 29 us; skew 500 ppm
I20260926 02:01:31.554082 32187 webserver.cc:533] Webserver started at http://127.29.89.189:44675/ using document root <none> and password file <none>
I20260926 02:01:31.554672 32187 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:31.554729 32187 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:31.557705 32187 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 02:01:31.559700 32201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:31.560774 32187 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 02:01:31.560901 32187 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
uuid: "d95dbe14730b406b8f4ffced8abe38f2"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:31.561260 32187 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:31.573886 32187 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:31.574455 32187 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:31.574630 32187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:31.581095 32187 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.189:33299
I20260926 02:01:31.581101 32253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.189:33299 every 8 connection(s)
I20260926 02:01:31.582027 32187 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-1/data/info.pb
I20260926 02:01:31.585705 32254 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:31.586565 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 32187
I20260926 02:01:31.587515 32254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap starting.
I20260926 02:01:31.593861 32254 log.cc:826] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:31.605499 32254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=21 overwritten=1 applied=20 ignored=0} inserts{seen=11 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:31.605865 32254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Bootstrap complete.
I20260926 02:01:31.608412 32254 raft_consensus.cc:359] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:31.608726 32254 raft_consensus.cc:740] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Initialized, Role: FOLLOWER
I20260926 02:01:31.609084 32254 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:31.609514 32258 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:31.609635 32258 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:31.610129 32254 sys_catalog.cc:565] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:31.615401 32272 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:31.615461 32272 catalog_manager.cc:1601] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: loading cluster ID for follower catalog manager: success
I20260926 02:01:31.617666 32272 catalog_manager.cc:1623] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: acquiring CA information for follower catalog manager: success
I20260926 02:01:31.619053 32272 catalog_manager.cc:1651] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:31.631358 32187 master_runner.cc:187] Error getting master registration for 127.29.89.188:46477: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111),
I20260926 02:01:31.633643 32187 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:31.768165 30748 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:31.770166 32218 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:40491; Asking this server to re-register.
I20260926 02:01:31.770665 30748 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:31.771759 32218 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:31.809564 31020 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:31.809717 31020 heartbeater.cc:499] Master 127.29.89.189:33299 was elected leader, sending a full tablet report...
I20260926 02:01:31.810382 30884 heartbeater.cc:344] Connected to a master server at 127.29.89.189:33299
I20260926 02:01:31.810544 32218 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:54931; Asking this server to re-register.
I20260926 02:01:31.811038 31020 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:31.811173 32218 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:59083; Asking this server to re-register.
I20260926 02:01:31.811600 30884 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:31.811878 32218 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:31.812398 32218 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:32.184070 32284 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:32.184309 32284 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:32.185576 32284 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:32.186079 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:32.186239 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 7.
W20260926 02:01:32.186379 32202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:32.186748 32202 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters:
I20260926 02:01:32.187004 32284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260926 02:01:32.187098 32284 raft_consensus.cc:493] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:32.187204 32284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 7 FOLLOWER]: Advancing to term 8
W20260926 02:01:32.187429 32202 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:32.189211 32284 raft_consensus.cc:515] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:32.189512 32284 leader_election.cc:290] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:32.189890 31908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d95dbe14730b406b8f4ffced8abe38f2" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:32.190056 31908 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 7 FOLLOWER]: Advancing to term 8
W20260926 02:01:32.190619 32202 leader_election.cc:336] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:32.191218 31908 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95dbe14730b406b8f4ffced8abe38f2 in term 8.
I20260926 02:01:32.191507 32202 leader_election.cc:304] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2; no voters: b285db155484443594c9385155358d4d
I20260926 02:01:32.191706 32284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 8 FOLLOWER]: Leader election won for term 8
I20260926 02:01:32.191869 32284 raft_consensus.cc:697] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [term 8 LEADER]: Becoming Leader. State: Replica: d95dbe14730b406b8f4ffced8abe38f2, State: Running, Role: LEADER
I20260926 02:01:32.192154 32284 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:32.193868 32287 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95dbe14730b406b8f4ffced8abe38f2. Latest consensus state: current_term: 8 leader_uuid: "d95dbe14730b406b8f4ffced8abe38f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:32.193997 32287 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:32.194404 32289 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:32.195843 32289 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:32.196777 32289 tablet_loader.cc:96] loaded metadata for tablet 134188f591ff45bbb46f452b4b018662 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:32.197480 32289 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:32.197999 32289 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:32.198055 32289 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:32.198670 32289 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:32.199015 32289 catalog_manager.cc:6230] T 00000000000000000000000000000000 P d95dbe14730b406b8f4ffced8abe38f2: Loaded TSK: 0
I20260926 02:01:32.199491 32289 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:32.199841 32289 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:32.200166 32289 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:32.230578 32218 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.230859 32214 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48140:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
I20260926 02:01:32.230971 32209 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.230880 32212 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231172 32217 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231220 32216 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231329 32211 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231384 32210 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231487 32215 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:32.231398 32213 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48140:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 02:01:32.238124 32271 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fc09b1f7d5d google::LogMessage::Fail() at ??:0
@ 0x7fc09b1fbb23 google::LogMessage::SendToLog() at ??:0
@ 0x7fc09b1f775c google::LogMessage::Flush() at ??:0
@ 0x7fc09b1f79a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fc0a07b8959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fc0a07b7870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fc0a07b6fd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
@ 0x7fc0a07bc6da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5569eb100f7e std::function<>::operator()() at ??:0
@ 0x7fc09c6d3c6a kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc09d0526db start_thread at ??:0
@ 0x7fc09a3ae61f clone at ??:0
W20260926 02:01:32.418709 30906 connection.cc:573] client connection to 127.29.89.189:33299 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:32.418676 30770 connection.cc:573] client connection to 127.29.89.189:33299 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:32.418751 30634 connection.cc:573] client connection to 127.29.89.189:33299 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 02:01:32.419548 31026 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.189:33299)
W20260926 02:01:32.419855 31026 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.189:33299)
W20260926 02:01:32.420257 31026 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.189:33299)
I20260926 02:01:33.458716 32307 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:33.458873 32307 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:33.459355 32307 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:33.460825 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:33.460916 31882 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:33.460981 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:33.461134 32307 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260926 02:01:34.743961 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--webserver_interface=127.29.89.188
--webserver_port=36293
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:34.854442 32312 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:34.854642 32312 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:34.854682 32312 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:34.858081 32312 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:34.858139 32312 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:34.858158 32312 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:34.858176 32312 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:34.862538 32312 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.188:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.89.188
--webserver_port=36293
--never_fsync=true
--heap_profile_path=/tmp/kudu.32312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:34.863703 32312 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:34.864812 32312 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:34.870601 32322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:34.870626 32319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:34.870638 32312 server_base.cc:1061] running on GCE node
W20260926 02:01:34.870659 32320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:34.871354 32312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:34.872210 32312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:34.873414 32312 hybrid_clock.cc:648] HybridClock initialized: now 1790388094873390 us; error 44 us; skew 500 ppm
I20260926 02:01:34.875180 32312 webserver.cc:533] Webserver started at http://127.29.89.188:36293/ using document root <none> and password file <none>
I20260926 02:01:34.875741 32312 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:34.875826 32312 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:34.879261 32312 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260926 02:01:34.881165 32328 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:34.882272 32312 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 02:01:34.882473 32312 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
uuid: "b285db155484443594c9385155358d4d"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:34.882886 32312 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:34.910075 32312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:34.910830 32312 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:34.911026 32312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:34.918494 32312 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.188:46477
I20260926 02:01:34.918537 32380 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.188:46477 every 8 connection(s)
I20260926 02:01:34.919601 32312 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-2/data/info.pb
I20260926 02:01:34.920171 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 32312
I20260926 02:01:34.923964 32381 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:34.926085 32381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap starting.
I20260926 02:01:34.931758 32381 log.cc:826] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:34.932586 31867 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:34.932693 31867 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:34.942776 32381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=17 ignored=0} inserts{seen=10 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260926 02:01:34.943256 32381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Bootstrap complete.
I20260926 02:01:34.946297 32381 raft_consensus.cc:359] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:34.946785 32381 raft_consensus.cc:740] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: b285db155484443594c9385155358d4d, State: Initialized, Role: FOLLOWER
I20260926 02:01:34.947124 32381 consensus_queue.cc:260] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 6.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:34.947479 32386 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:34.947603 32386 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:34.948041 32381 sys_catalog.cc:565] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:34.953060 32400 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:34.953131 32400 catalog_manager.cc:1601] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: loading cluster ID for follower catalog manager: success
I20260926 02:01:34.955024 32400 catalog_manager.cc:1623] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: acquiring CA information for follower catalog manager: success
I20260926 02:01:34.956303 32400 catalog_manager.cc:1651] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:34.959251 32312 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:34.961275 32312 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:35.059356 32405 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:35.059546 32405 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:35.060077 32405 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:35.061674 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:35.064543 32355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d" is_pre_election: true
I20260926 02:01:35.064841 32355 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 995d22fd6e1b4cdeb3b8aa4c61b56272 in term 6.
I20260926 02:01:35.065217 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:35.065423 32405 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260926 02:01:35.065539 32405 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:35.065591 32405 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 8 FOLLOWER]: Advancing to term 9
I20260926 02:01:35.066555 32405 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:35.066933 32405 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 election: Requested vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
I20260926 02:01:35.067322 32355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: false dest_uuid: "b285db155484443594c9385155358d4d"
I20260926 02:01:35.067453 32355 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 6 FOLLOWER]: Advancing to term 9
W20260926 02:01:35.067842 31883 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:35.069692 32355 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 995d22fd6e1b4cdeb3b8aa4c61b56272 in term 9.
I20260926 02:01:35.069993 31882 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:35.070228 32405 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Leader election won for term 9
I20260926 02:01:35.070454 32405 raft_consensus.cc:697] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 LEADER]: Becoming Leader. State: Replica: 995d22fd6e1b4cdeb3b8aa4c61b56272, State: Running, Role: LEADER
I20260926 02:01:35.070767 32405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:35.071988 32408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995d22fd6e1b4cdeb3b8aa4c61b56272. Latest consensus state: current_term: 9 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:35.072095 32408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:35.072427 32410 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:35.074210 32410 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:35.075385 32410 tablet_loader.cc:96] loaded metadata for tablet 134188f591ff45bbb46f452b4b018662 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:35.076040 32410 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:35.076553 32410 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:35.076619 32410 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:35.077260 32410 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:35.077723 32410 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Loaded TSK: 0
I20260926 02:01:35.078403 32410 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:35.078869 32410 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:35.079334 32410 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:35.092296 31897 master_service.cc:960] ReplaceTablet: received request to replace tablet 134188f591ff45bbb46f452b4b018662 from {username='slave'} at 127.0.0.1:49056
I20260926 02:01:35.092551 31898 master_service.cc:960] ReplaceTablet: received request to replace tablet 134188f591ff45bbb46f452b4b018662 from {username='slave'} at 127.0.0.1:49056
I20260926 02:01:35.092639 31896 master_service.cc:960] ReplaceTablet: received request to replace tablet 134188f591ff45bbb46f452b4b018662 from {username='slave'} at 127.0.0.1:49056
I20260926 02:01:35.092679 31895 master_service.cc:960] ReplaceTablet: received request to replace tablet 134188f591ff45bbb46f452b4b018662 from {username='slave'} at 127.0.0.1:49056
I20260926 02:01:35.093127 31894 master_service.cc:960] ReplaceTablet: received request to replace tablet 134188f591ff45bbb46f452b4b018662 from {username='slave'} at 127.0.0.1:49056
I20260926 02:01:35.093537 31891 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.093791 31890 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.094020 31892 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.094033 31889 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.094262 31893 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_aba582f363ee48b38f7b72fe1507952f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.096340 32355 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Refusing update from remote peer 995d22fd6e1b4cdeb3b8aa4c61b56272: Log matching property violated. Preceding OpId in replica: term: 6 index: 18. Preceding OpId from leader: term: 9 index: 22. (index mismatch)
I20260926 02:01:35.096903 32408 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.000s
I20260926 02:01:35.097033 32408 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [LEADER]: Peer b285db155484443594c9385155358d4d log is divergent from this leader: its last log entry 6.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
W20260926 02:01:35.097386 31883 consensus_peers.cc:597] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 -> Peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Couldn't send request to peer d95dbe14730b406b8f4ffced8abe38f2. Status: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 02:01:35.098013 32355 pending_rounds.cc:85] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Aborting all ops after (but not including) 17
I20260926 02:01:35.098094 32355 pending_rounds.cc:107] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Aborting uncommitted NO_OP operation due to leader change: 6.18
I20260926 02:01:35.098160 32355 raft_consensus.cc:2889] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260926 02:01:35.101507 32386 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995d22fd6e1b4cdeb3b8aa4c61b56272. Latest consensus state: current_term: 9 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:35.101610 32386 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:35.101689 32386 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:35.101737 32386 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:35.103480 32412 mvcc.cc:204] Tried to move back new op lower bound from 7333429614623920128 to 7333429614331195392. Current Snapshot: MvccSnapshot[applied={T|T < 7333429614623920128 or (T in {7333429614623920128})}]
F20260926 02:01:35.104458 31953 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260926 02:01:35.106206 32405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:35.106361 32405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:35.106647 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:35.106758 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f544ca90d5d google::LogMessage::Fail() at ??:0
I20260926 02:01:35.109470 31897 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Sending DeleteTablet for 3 replicas of tablet 134188f591ff45bbb46f452b4b018662
I20260926 02:01:35.109535 31894 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f544ca94b23 google::LogMessage::SendToLog() at ??:0
I20260926 02:01:35.110680 31897 catalog_manager.cc:6934] ReplaceTablet: tablet 134188f591ff45bbb46f452b4b018662 deleted and replaced by tablet 966c204ae51a49a9adc575cb768f384e
I20260926 02:01:35.110891 31897 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.111330 31898 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.112007 31896 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49056:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:35.112704 31895 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49056:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
@ 0x7f544ca9075c google::LogMessage::Flush() at ??:0
@ 0x7f544ca909a1 google::LogMessage::~LogMessage() at ??:0
I20260926 02:01:35.122905 30951 tablet_service.cc:1558] Processing DeleteTablet for tablet 134188f591ff45bbb46f452b4b018662 with delete_type TABLET_DATA_DELETED (replaced by tablet 966c204ae51a49a9adc575cb768f384e) from {username='slave'} at 127.0.0.1:53638
I20260926 02:01:35.123140 30952 tablet_service.cc:1511] Processing CreateTablet for tablet 966c204ae51a49a9adc575cb768f384e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:35.123507 30952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966c204ae51a49a9adc575cb768f384e. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5452051959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 02:01:35.127127 30679 tablet_service.cc:1558] Processing DeleteTablet for tablet 134188f591ff45bbb46f452b4b018662 with delete_type TABLET_DATA_DELETED (replaced by tablet 966c204ae51a49a9adc575cb768f384e) from {username='slave'} at 127.0.0.1:33324
I20260926 02:01:35.127554 30680 tablet_service.cc:1511] Processing CreateTablet for tablet 966c204ae51a49a9adc575cb768f384e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:35.127810 30680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966c204ae51a49a9adc575cb768f384e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:35.129446 32425 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe: Bootstrap starting.
I20260926 02:01:35.129597 30816 tablet_service.cc:1558] Processing DeleteTablet for tablet 134188f591ff45bbb46f452b4b018662 with delete_type TABLET_DATA_DELETED (replaced by tablet 966c204ae51a49a9adc575cb768f384e) from {username='slave'} at 127.0.0.1:41476
I20260926 02:01:35.130029 30815 tablet_service.cc:1511] Processing CreateTablet for tablet 966c204ae51a49a9adc575cb768f384e (DEFAULT_TABLE table=default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 02:01:35.130242 32424 tablet_replica.cc:333] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: stopping tablet replica
I20260926 02:01:35.130290 30815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966c204ae51a49a9adc575cb768f384e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 02:01:35.130371 32424 raft_consensus.cc:2243] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:35.130493 32424 raft_consensus.cc:2272] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:35.130558 32425 tablet_bootstrap.cc:654] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:35.130883 32424 ts_tablet_manager.cc:1916] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f5452050870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 02:01:35.133031 32424 ts_tablet_manager.cc:1929] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:35.133093 32424 log.cc:1199] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-2/wal/wals/134188f591ff45bbb46f452b4b018662
I20260926 02:01:35.133306 32430 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07: Bootstrap starting.
I20260926 02:01:35.133353 32424 ts_tablet_manager.cc:1950] T 134188f591ff45bbb46f452b4b018662 P 02a024f3eff64b34b0fa7e46d6274abe: Deleting consensus metadata
I20260926 02:01:35.133605 32427 tablet_replica.cc:333] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: stopping tablet replica
I20260926 02:01:35.133704 32427 raft_consensus.cc:2243] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 02:01:35.133797 32427 raft_consensus.cc:2272] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:35.134197 32427 ts_tablet_manager.cc:1916] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:35.134652 32430 tablet_bootstrap.cc:654] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:35.135404 32426 tablet_replica.cc:333] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: stopping tablet replica
I20260926 02:01:35.135511 32426 raft_consensus.cc:2243] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 LEADER]: Raft consensus shutting down.
I20260926 02:01:35.135741 32430 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07: No bootstrap required, opened a new log
I20260926 02:01:35.135811 32430 ts_tablet_manager.cc:1403] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 02:01:35.135826 32426 raft_consensus.cc:2272] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 02:01:35.136148 32427 ts_tablet_manager.cc:1929] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:35.136184 32426 ts_tablet_manager.cc:1916] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 02:01:35.136205 32427 log.cc:1199] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-1/wal/wals/134188f591ff45bbb46f452b4b018662
I20260926 02:01:35.136266 32430 raft_consensus.cc:359] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.136358 32430 raft_consensus.cc:385] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:35.136399 32430 raft_consensus.cc:740] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fbed94c9e248d0a89843d39b01ef07, State: Initialized, Role: FOLLOWER
I20260926 02:01:35.136449 32427 ts_tablet_manager.cc:1950] T 134188f591ff45bbb46f452b4b018662 P 08fbed94c9e248d0a89843d39b01ef07: Deleting consensus metadata
I20260926 02:01:35.136533 32430 consensus_queue.cc:260] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.136860 32430 ts_tablet_manager.cc:1434] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 02:01:35.137542 31883 catalog_manager.cc:5063] TS 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271): tablet 134188f591ff45bbb46f452b4b018662 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:35.138226 32426 ts_tablet_manager.cc:1929] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 02:01:35.138288 32426 log.cc:1199] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting WAL directory at /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/ts-0/wal/wals/134188f591ff45bbb46f452b4b018662
I20260926 02:01:35.138571 32426 ts_tablet_manager.cc:1950] T 134188f591ff45bbb46f452b4b018662 P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Deleting consensus metadata
@ 0x7f545204ffd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
I20260926 02:01:35.140323 32428 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Bootstrap starting.
I20260926 02:01:35.141134 31884 catalog_manager.cc:5063] TS 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981): tablet 134188f591ff45bbb46f452b4b018662 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:35.141469 32428 tablet_bootstrap.cc:654] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Neither blocks nor log segments found. Creating new log.
I20260926 02:01:35.142330 30882 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:35.142663 32425 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe: No bootstrap required, opened a new log
I20260926 02:01:35.142752 32425 ts_tablet_manager.cc:1403] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260926 02:01:35.143491 32425 raft_consensus.cc:359] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.143612 32425 raft_consensus.cc:385] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:35.143649 32425 raft_consensus.cc:740] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Initialized, Role: FOLLOWER
I20260926 02:01:35.143842 32425 consensus_queue.cc:260] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.144260 32425 ts_tablet_manager.cc:1434] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260926 02:01:35.144418 32345 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:55355; Asking this server to re-register.
I20260926 02:01:35.146019 32428 tablet_bootstrap.cc:492] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: No bootstrap required, opened a new log
I20260926 02:01:35.146101 32428 ts_tablet_manager.cc:1403] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260926 02:01:35.146507 30882 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:35.146706 32428 raft_consensus.cc:359] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.146822 32428 raft_consensus.cc:385] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 02:01:35.146862 32428 raft_consensus.cc:740] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a655024b1b40dbbc5dc2cf5d6dd6fb, State: Initialized, Role: FOLLOWER
I20260926 02:01:35.147009 32428 consensus_queue.cc:260] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
@ 0x7f54520556da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
I20260926 02:01:35.147334 32428 ts_tablet_manager.cc:1434] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
@ 0x55a89f4e3f7e std::function<>::operator()() at ??:0
I20260926 02:01:35.150959 32345 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:35.152535 31885 catalog_manager.cc:5063] TS 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777): tablet 134188f591ff45bbb46f452b4b018662 (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]) successfully deleted
I20260926 02:01:35.153156 31018 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:35.153338 30746 heartbeater.cc:344] Connected to a master server at 127.29.89.188:46477
I20260926 02:01:35.154183 32344 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:39023; Asking this server to re-register.
I20260926 02:01:35.154773 31018 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:35.154914 32344 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:57771; Asking this server to re-register.
@ 0x7f544df6cc6a kudu::Thread::SuperviseThread() at ??:0
I20260926 02:01:35.155565 30746 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:35.156179 32344 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:35.156495 32345 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
@ 0x7f544e8eb6db start_thread at ??:0
@ 0x7f544bc4761f clone at ??:0
I20260926 02:01:35.236253 32436 raft_consensus.cc:493] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:35.236449 32436 raft_consensus.cc:515] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.237123 32436 leader_election.cc:290] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:35.237656 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966c204ae51a49a9adc575cb768f384e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07" is_pre_election: true
I20260926 02:01:35.237653 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966c204ae51a49a9adc575cb768f384e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" is_pre_election: true
I20260926 02:01:35.237811 30836 raft_consensus.cc:2468] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:35.237811 30699 raft_consensus.cc:2468] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 0.
I20260926 02:01:35.238183 30908 leader_election.cc:304] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 08fbed94c9e248d0a89843d39b01ef07; no voters:
I20260926 02:01:35.238402 32436 raft_consensus.cc:2804] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 02:01:35.238478 32436 raft_consensus.cc:493] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 02:01:35.238518 32436 raft_consensus.cc:3060] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:35.239627 32436 raft_consensus.cc:515] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
I20260926 02:01:35.240003 32436 leader_election.cc:290] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Requested vote from peers 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777), 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:35.240360 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966c204ae51a49a9adc575cb768f384e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb"
I20260926 02:01:35.240358 30836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966c204ae51a49a9adc575cb768f384e" candidate_uuid: "02a024f3eff64b34b0fa7e46d6274abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08fbed94c9e248d0a89843d39b01ef07"
I20260926 02:01:35.240504 30699 raft_consensus.cc:3060] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:35.240474 30836 raft_consensus.cc:3060] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 02:01:35.241319 30699 raft_consensus.cc:2468] T 966c204ae51a49a9adc575cb768f384e P 92a655024b1b40dbbc5dc2cf5d6dd6fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:35.241521 30836 raft_consensus.cc:2468] T 966c204ae51a49a9adc575cb768f384e P 08fbed94c9e248d0a89843d39b01ef07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02a024f3eff64b34b0fa7e46d6274abe in term 1.
I20260926 02:01:35.241639 30907 leader_election.cc:304] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a024f3eff64b34b0fa7e46d6274abe, 92a655024b1b40dbbc5dc2cf5d6dd6fb; no voters:
I20260926 02:01:35.241869 32436 raft_consensus.cc:2804] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 02:01:35.242050 32436 raft_consensus.cc:697] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [term 1 LEADER]: Becoming Leader. State: Replica: 02a024f3eff64b34b0fa7e46d6274abe, State: Running, Role: LEADER
I20260926 02:01:35.242230 32436 consensus_queue.cc:237] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 } } peers { permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 } } peers { permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" member_type: VOTER last_known_addr { host: "127.29.89.131" port: 36271 } }
W20260926 02:01:35.307031 31025 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.190:45707)
W20260926 02:01:35.307196 31025 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.190:45707)
W20260926 02:01:35.311553 31025 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.190:45707)
I20260926 02:01:35.617156 32450 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:35.617273 32450 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:35.618511 32450 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:35.619002 32329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:35.619755 32329 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:35.619872 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:35.619957 32330 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:35.620150 32450 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 02:01:35.737167 32436 consensus_queue.cc:1048] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" member_type: VOTER last_known_addr { host: "127.29.89.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:35.740620 32448 consensus_queue.cc:1048] T 966c204ae51a49a9adc575cb768f384e P 02a024f3eff64b34b0fa7e46d6274abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" member_type: VOTER last_known_addr { host: "127.29.89.130" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 02:01:36.136722 32465 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:36.136855 32465 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:36.137427 32465 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:36.138717 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:36.138824 32329 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:36.138893 32329 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:36.139027 32465 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 02:01:36.821120 32480 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:36.821350 32480 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:36.821902 32480 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:36.823098 32329 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
W20260926 02:01:36.823252 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:36.823323 32330 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:36.823469 32480 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 02:01:37.378679 32492 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:37.378858 32492 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:37.379348 32492 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:37.380614 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:37.380760 32329 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707): Network error: Client connection negotiation failed: client connection to 127.29.89.190:45707: connect: Connection refused (error 111)
I20260926 02:01:37.380865 32329 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b285db155484443594c9385155358d4d; no voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:37.380996 32492 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 02:01:37.749037 30054 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
/tmp/dist-test-taskMwEE2d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--webserver_interface=127.29.89.190
--webserver_port=41125
--builtin_ntp_servers=127.29.89.148:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 02:01:37.852954 32500 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 02:01:37.853160 32500 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 02:01:37.853200 32500 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 02:01:37.856510 32500 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 02:01:37.856562 32500 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 02:01:37.856582 32500 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 02:01:37.856599 32500 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 02:01:37.860445 32500 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.89.148:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:33797
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.89.190:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.89.190
--webserver_port=41125
--never_fsync=true
--heap_profile_path=/tmp/kudu.32500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type DEBUG
built by None at 26 Sep 2026 01:43:13 UTC on bdcb31816ec0
build id 13999
I20260926 02:01:37.861403 32500 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 02:01:37.862531 32500 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 02:01:37.868419 32507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:37.868494 32500 server_base.cc:1061] running on GCE node
W20260926 02:01:37.868516 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 02:01:37.868430 32510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 02:01:37.869062 32500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 02:01:37.869925 32500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 02:01:37.871114 32500 hybrid_clock.cc:648] HybridClock initialized: now 1790388097871098 us; error 31 us; skew 500 ppm
I20260926 02:01:37.872794 32500 webserver.cc:533] Webserver started at http://127.29.89.190:41125/ using document root <none> and password file <none>
I20260926 02:01:37.873404 32500 fs_manager.cc:362] Metadata directory not provided
I20260926 02:01:37.873474 32500 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 02:01:37.876674 32500 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260926 02:01:37.878546 32516 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 02:01:37.879477 32500 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 02:01:37.879603 32500 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data,/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
format_stamp: "Formatted at 2026-09-26 02:01:07 on dist-test-slave-1hsp"
I20260926 02:01:37.879964 32500 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 02:01:37.903667 32500 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 02:01:37.904322 32500 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 02:01:37.904572 32500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 02:01:37.912273 32568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.89.190:45707 every 8 connection(s)
I20260926 02:01:37.912298 32500 rpc_server.cc:307] RPC server started. Bound to: 127.29.89.190:45707
I20260926 02:01:37.913254 32500 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/master-0/data/info.pb
I20260926 02:01:37.914788 30054 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskMwEE2d/build/debug/bin/kudu as pid 32500
I20260926 02:01:37.917371 32569 sys_catalog.cc:263] Verifying existing consensus state
I20260926 02:01:37.919657 32569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap starting.
I20260926 02:01:37.925870 32569 log.cc:826] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Log is configured to *not* fsync() on all Append() calls
I20260926 02:01:37.938195 32569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=12 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 02:01:37.938599 32569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: Bootstrap complete.
I20260926 02:01:37.941045 32569 raft_consensus.cc:359] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:37.941337 32569 raft_consensus.cc:740] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 995d22fd6e1b4cdeb3b8aa4c61b56272, State: Initialized, Role: FOLLOWER
I20260926 02:01:37.941754 32569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 9.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:37.942229 32573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:37.942382 32573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 02:01:37.942818 32569 sys_catalog.cc:565] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 02:01:37.947907 32587 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:37.948066 32587 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: loading cluster ID for follower catalog manager: success
I20260926 02:01:37.950663 32587 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: acquiring CA information for follower catalog manager: success
I20260926 02:01:37.952533 32587 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 02:01:37.953094 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:37.955103 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:38.251597 30747 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:38.251981 30883 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:38.253337 32533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "08fbed94c9e248d0a89843d39b01ef07" instance_seqno: 1790388068702997) as {username='slave'} at 127.29.89.130:48739; Asking this server to re-register.
I20260926 02:01:38.253337 32532 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "92a655024b1b40dbbc5dc2cf5d6dd6fb" instance_seqno: 1790388068536003) as {username='slave'} at 127.29.89.129:43183; Asking this server to re-register.
I20260926 02:01:38.253849 30883 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:38.253865 30747 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:38.255126 32533 ts_manager.cc:194] Registered new tserver with Master: 92a655024b1b40dbbc5dc2cf5d6dd6fb (127.29.89.129:38981)
I20260926 02:01:38.255157 32532 ts_manager.cc:194] Registered new tserver with Master: 08fbed94c9e248d0a89843d39b01ef07 (127.29.89.130:38777)
I20260926 02:01:38.302347 32596 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:38.302493 32596 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:38.303114 32596 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
W20260926 02:01:38.304478 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:38.306767 32543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 10 candidate_status { last_received { term: 9 index: 23 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" is_pre_election: true
I20260926 02:01:38.307097 32543 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 9.
I20260926 02:01:38.307477 32329 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:38.307657 32596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260926 02:01:38.307719 32596 raft_consensus.cc:493] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 995d22fd6e1b4cdeb3b8aa4c61b56272)
I20260926 02:01:38.307760 32596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 9 FOLLOWER]: Advancing to term 10
I20260926 02:01:38.308614 32596 raft_consensus.cc:515] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:38.308970 32596 leader_election.cc:290] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 election: Requested vote from peers 995d22fd6e1b4cdeb3b8aa4c61b56272 (127.29.89.190:45707), d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299)
I20260926 02:01:38.309281 32543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b285db155484443594c9385155358d4d" candidate_term: 10 candidate_status { last_received { term: 9 index: 23 } } ignore_live_leader: false dest_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272"
I20260926 02:01:38.309422 32543 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 9 FOLLOWER]: Advancing to term 10
W20260926 02:01:38.309966 32330 leader_election.cc:336] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:38.311352 32543 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b285db155484443594c9385155358d4d in term 10.
I20260926 02:01:38.311672 32329 leader_election.cc:304] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272, b285db155484443594c9385155358d4d; no voters: d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:38.311882 32596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 10 FOLLOWER]: Leader election won for term 10
I20260926 02:01:38.312108 32596 raft_consensus.cc:697] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [term 10 LEADER]: Becoming Leader. State: Replica: b285db155484443594c9385155358d4d, State: Running, Role: LEADER
I20260926 02:01:38.312448 32596 consensus_queue.cc:237] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 9.23, Last appended by leader: 23, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:38.313556 32600 sys_catalog.cc:455] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b285db155484443594c9385155358d4d. Latest consensus state: current_term: 10 leader_uuid: "b285db155484443594c9385155358d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } } }
I20260926 02:01:38.313669 32600 sys_catalog.cc:458] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d [sys.catalog]: This master's current role is: LEADER
I20260926 02:01:38.313961 32602 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 02:01:38.315466 32602 catalog_manager.cc:692] Loaded metadata for table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284]
I20260926 02:01:38.316677 32602 tablet_loader.cc:96] loaded metadata for tablet 966c204ae51a49a9adc575cb768f384e (table default.table_44705ce850b5424984beb85e0103c16c [id=eb0615a428ac43c0abb1a2c982f10284])
I20260926 02:01:38.317101 32602 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 02:01:38.317456 32602 catalog_manager.cc:1308] Loaded cluster ID: e346fec5c0bb4d99bb41c6b7cc263b33
I20260926 02:01:38.317500 32602 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 02:01:38.318099 32602 catalog_manager.cc:1553] Loading token signing keys...
I20260926 02:01:38.318536 32602 catalog_manager.cc:6230] T 00000000000000000000000000000000 P b285db155484443594c9385155358d4d: Loaded TSK: 0
I20260926 02:01:38.319168 32602 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 02:01:38.319577 32602 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 02:01:38.320099 32602 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 5
I20260926 02:01:38.346414 31019 heartbeater.cc:344] Connected to a master server at 127.29.89.190:45707
I20260926 02:01:38.347309 32533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "02a024f3eff64b34b0fa7e46d6274abe" instance_seqno: 1790388068881274) as {username='slave'} at 127.29.89.131:53393; Asking this server to re-register.
I20260926 02:01:38.347713 31019 heartbeater.cc:461] Registering TS with master...
I20260926 02:01:38.348585 32533 ts_manager.cc:194] Registered new tserver with Master: 02a024f3eff64b34b0fa7e46d6274abe (127.29.89.131:36271)
I20260926 02:01:38.357120 32345 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_8917cd5381bb470e8ee76b412f3ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357236 32344 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50924:
table { table_name: "default.Table_44705ce850b5424984beb85e0103c16c" } new_table_name: "default.Table_40e7cbf4d2a34a698b15008eb41c23a7" modify_external_catalogs: true
I20260926 02:01:38.357414 32340 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_6cdde7bca9a8417091fa686ec9494cd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357563 32342 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_095de6750e1242cfb2364d77e2b4deb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357611 32339 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357781 32336 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_67533d3acf714eba8041a0a5675dd2eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357816 32337 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_fc856879f4024619ba0ceaa95ae58ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357841 32343 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_3e297a6e8a69461b8c6d85132495df8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.357990 32341 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_351548b11fca4255832809bbaf8f8a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 02:01:38.358059 32338 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50924:
name: "default.Table_28bfb85f5dab4406b28b8dc939814d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 02:01:38.363832 32399 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 6, previous 6; dumping first 2 out of 16 received events: NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_44705ce850b5424984beb85e0103c16c, message={"server":"","servicePrincipal":"","db":"default","table":"table_44705ce850b5424984beb85e0103c16c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_44705ce850b5424984beb85e0103c16c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_44705ce850b5424984beb85e0103c16c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_44705ce850b5424984beb85e0103c16c\",\"kudu.table_id\":\"eb0615a428ac43c0abb1a2c982f10284\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=6, eventTime=1790388070, eventType=CREATE_TABLE, dbName=default, tableName=table_351548b11fca4255832809bbaf8f8a19, message={"server":"","servicePrincipal":"","db":"default","table":"table_351548b11fca4255832809bbaf8f8a19","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_351548b11fca4255832809bbaf8f8a19\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790388070},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskMwEE2d/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790388052049196-30054-0/minicluster-data/warehouse/table_351548b11fca4255832809bbaf8f8a19\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_351548b11fca4255832809bbaf8f8a19\",\"kudu.table_id\":\"d6b5e0e0086e42698f03eb96e3becabb\",\"transient_lastDdlTime\":\"1790388070\",\"comment\":\"\",\"kudu.master_addresses\":\"127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"e346fec5c0bb4d99bb41c6b7cc263b33\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790388070,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fae8642bd5d google::LogMessage::Fail() at ??:0
@ 0x7fae8642fb23 google::LogMessage::SendToLog() at ??:0
@ 0x7fae8642b75c google::LogMessage::Flush() at ??:0
@ 0x7fae8642b9a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fae8b9ec959 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fae8b9eb870 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fae8b9eafd3 _ZZN4kudu6master30HmsNotificationLogListenerTask4InitEvENKUlvE_clEv at ??:0
@ 0x7fae8b9f06da _ZNSt17_Function_handlerIFvvEZN4kudu6master30HmsNotificationLogListenerTask4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x561cfd717f7e std::function<>::operator()() at ??:0
@ 0x7fae87907c6a kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae882866db start_thread at ??:0
@ 0x7fae855e261f clone at ??:0
W20260926 02:01:38.550444 31025 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.29.89.188:46477)
W20260926 02:01:38.550778 31025 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.89.188:46477)
W20260926 02:01:38.551076 31025 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.29.89.188:46477)
I20260926 02:01:38.845633 31018 heartbeater.cc:499] Master 127.29.89.188:46477 was elected leader, sending a full tablet report...
I20260926 02:01:39.006150 32618 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:39.006283 32618 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:39.007529 32618 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:39.008126 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:39.008980 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:39.009186 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:39.009282 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:39.009485 32618 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:39.520087 32622 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:39.520207 32622 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:39.520748 32622 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:39.522053 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:39.522156 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:39.522254 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:39.522431 32622 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:40.293154 30054 master-stress-test.cc:579] Killing test thread 1/25
I20260926 02:01:40.476169 32628 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:40.476292 32628 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:40.476792 32628 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:40.477988 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:40.478086 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:40.478168 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:40.478279 32628 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:42.105252 32644 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:42.105408 32644 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:42.106005 32644 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:42.107653 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:42.107805 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:42.107892 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:42.108011 32644 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:42.956930 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:42.957041 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:44.520895 32675 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:44.521025 32675 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:44.521476 32675 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:44.522027 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260926 02:01:44.522794 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:44.522892 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:44.522974 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:44.523149 32675 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:45.182731 32685 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:45.182915 32685 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:45.183408 32685 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:45.184696 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:45.184799 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:45.184904 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:45.185055 32685 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:47.959019 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:47.959116 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:49.933964 32746 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:49.934082 32746 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:49.934624 32746 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:49.935113 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:01:49.935879 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:01:49.935971 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:01:49.936053 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:49.936219 32746 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:52.961251 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:52.961380 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:01:53.843361 326 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:01:53.843497 326 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:01:53.844096 326 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:01:53.845634 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:01:53.845741 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:01:53.845805 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:01:53.845957 326 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:01:57.963392 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:01:57.963543 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:01.516080 405 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:01.516228 405 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:01.516645 405 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:01.517333 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:02:01.518085 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:02:01.518180 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:02:01.518226 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:01.518395 405 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:02.965840 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:02.965958 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:06.387670 461 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:06.387794 461 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:06.388185 461 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:06.389474 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:02:06.389586 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:02:06.389663 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:06.389873 461 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:07.967942 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:07.968086 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:02:09.316777 30748 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (77 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:02:09.316905 30884 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (76 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:02:09.410053 31018 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (61 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:02:12.970095 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:12.970203 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:15.140784 549 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:15.140921 549 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:15.141453 549 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:15.142062 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:02:15.142910 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:02:15.142994 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:02:15.143055 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:15.143203 549 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:17.972345 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:17.972471 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:22.974363 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:22.974493 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:27.979624 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:27.979773 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:32.982074 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:32.982219 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:34.255445 777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:34.255609 777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:34.256330 777 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:34.256910 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:02:34.257917 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:02:34.258109 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:02:34.258204 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:34.258406 777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:37.984153 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:37.984295 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:42.986096 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:42.986239 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:47.172749 923 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:47.172885 923 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:47.173452 923 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:47.174078 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:02:47.174857 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:02:47.175007 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:02:47.175102 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:47.175267 923 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:47.988251 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:47.988379 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:52.990105 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:52.990206 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:02:55.599761 1023 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:02:55.599895 1023 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:02:55.600339 1023 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:02:55.600865 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:02:55.601550 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:02:55.601660 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:02:55.601737 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:02:55.601864 1023 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:02:57.992311 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:02:57.992430 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:02.995213 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:02.995342 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:07.997282 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:07.997377 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:03:09.454250 30882 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (181 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:03:09.454396 30748 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (197 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:03:09.539039 31018 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (181 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:03:10.558507 1192 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:10.558667 1192 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:10.559212 1192 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:10.559760 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:03:10.560549 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:03:10.560704 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:03:10.560763 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:10.560920 1192 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:12.999383 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:12.999485 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:14.730839 1246 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:14.730973 1246 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:14.731415 1246 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:14.732581 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:03:14.732707 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:03:14.732786 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:14.732944 1246 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:18.001525 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:18.001669 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:23.004179 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:23.004354 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:27.084455 1397 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:27.084585 1397 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:27.085067 1397 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:27.085531 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:03:27.086268 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:03:27.086418 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:03:27.086540 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:27.086722 1397 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:28.006623 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:28.006735 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:30.303007 1434 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:30.303172 1434 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:30.303634 1434 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:30.305032 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:03:30.305105 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:03:30.305147 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:30.305292 1434 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:33.008919 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:33.009048 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:37.288101 1521 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:37.288232 1521 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:37.288693 1521 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:37.289244 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:03:37.290002 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:03:37.290117 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:03:37.290218 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:37.290396 1521 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:38.011377 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:38.011515 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:38.059386 32522 maintenance_manager.cc:419] P 995d22fd6e1b4cdeb3b8aa4c61b56272: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033366
I20260926 02:03:38.093050 32521 maintenance_manager.cc:643] P 995d22fd6e1b4cdeb3b8aa4c61b56272: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.015s sys 0.000s Metrics: {"bytes_written":8512,"cfile_init":1,"dirs.queue_time_us":199,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":896,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":485,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":11,"thread_start_us":157,"threads_started":2,"wal-append.queue_time_us":150}
I20260926 02:03:38.094007 32522 maintenance_manager.cc:419] P 995d22fd6e1b4cdeb3b8aa4c61b56272: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 727 bytes on disk
I20260926 02:03:38.094640 32521 maintenance_manager.cc:643] P 995d22fd6e1b4cdeb3b8aa4c61b56272: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260926 02:03:39.249603 1551 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:39.249734 1551 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:39.250240 1551 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:39.251638 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:03:39.251719 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:03:39.251771 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:39.251912 1551 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:41.168386 1578 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:41.168533 1578 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:41.168977 1578 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:41.170290 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:03:41.170410 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:03:41.170469 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:41.170614 1578 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:43.013800 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:43.013932 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:48.016287 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:48.016404 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:53.018498 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:53.018601 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:03:54.201236 1776 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:03:54.201417 1776 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:03:54.201987 1776 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:03:54.202565 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260926 02:03:54.203279 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:03:54.203385 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:03:54.203471 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:03:54.203627 1776 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:03:58.020752 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:03:58.020903 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:03.023166 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:03.023298 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:08.027122 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:08.027230 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:04:09.593168 30884 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (316 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:04:09.593278 30746 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (301 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:04:09.670483 31018 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (301 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:04:11.678251 2045 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:11.678445 2045 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:11.679013 2045 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:11.679518 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:04:11.680174 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:04:11.680321 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:04:11.680395 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:11.680539 2045 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:04:13.029382 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:13.029486 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:18.031723 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:18.031888 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:19.778015 2168 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:19.778143 2168 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:19.778698 2168 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:19.779343 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:04:19.780129 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:04:19.780225 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:04:19.780308 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:19.780431 2168 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:04:23.033977 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:23.034082 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:28.036463 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:28.036598 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:33.039054 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:33.039213 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:33.041358 2367 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:33.041488 2367 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:33.041936 2367 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:33.042270 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:04:33.043022 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:04:33.043159 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:04:33.043236 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:33.043380 2367 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:04:38.041601 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:38.041709 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:39.466975 2468 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:39.467103 2468 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:39.467538 2468 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:39.468084 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:04:39.468778 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:04:39.468921 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:04:39.468998 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:39.469143 2468 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:04:43.043949 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:43.044081 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:48.046113 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:48.046252 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:53.048434 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:53.048540 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:56.594230 2722 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:56.594409 2722 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:56.595028 2722 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:56.595512 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:04:56.596257 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:04:56.596395 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:04:56.596473 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:56.596648 2722 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:04:58.050863 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:04:58.050971 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:04:59.713886 2773 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:04:59.714006 2773 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:04:59.714455 2773 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:04:59.715597 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:04:59.715724 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:04:59.715806 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:04:59.715991 2773 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:01.472054 2803 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:05:01.472187 2803 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:05:01.472776 2803 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:05:01.474110 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:05:01.474200 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:05:01.474272 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:05:01.474474 2803 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:03.053040 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:03.053150 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:08.055277 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:08.055382 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 02:05:09.731050 30882 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (421 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:05:09.731077 30748 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (437 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:05:09.802233 31020 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (438 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:05:12.632477 2971 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:05:12.632616 2971 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:05:12.633044 2971 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:05:12.633579 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260926 02:05:12.634333 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:05:12.634485 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:05:12.634559 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:05:12.634694 2971 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:13.057322 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:13.057430 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:13.264025 2985 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:05:13.264171 2985 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:05:13.264551 2985 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:05:13.265770 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:05:13.265877 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:05:13.265929 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:05:13.266073 2985 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:18.059538 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:18.059643 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:23.061506 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:23.061623 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:28.063926 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:28.064044 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:31.475283 3252 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:05:31.475409 3252 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:05:31.476037 3252 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:05:31.476666 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 02:05:31.477478 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:05:31.477556 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:05:31.477622 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:05:31.477741 3252 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:33.066171 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:33.066294 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:38.068523 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:38.068634 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:40.383931 3386 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:05:40.384121 3386 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:05:40.384626 3386 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:05:40.385155 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:05:40.385921 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:05:40.386102 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:05:40.386168 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:05:40.386353 3386 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:05:43.071041 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:43.071161 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:48.073418 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:48.073529 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:53.075528 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:53.075645 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:05:58.077549 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:05:58.077649 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:00.337543 3662 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:06:00.337728 3662 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:06:00.338426 3662 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:06:00.338960 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:06:00.339699 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:06:00.339809 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:06:00.339911 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:06:00.340071 3662 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:06:03.079909 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:03.080029 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:08.082160 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:08.082289 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:09.538422 3795 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:06:09.538602 3795 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:06:09.539100 3795 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:06:09.539700 32517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:06:09.540406 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:06:09.540567 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:06:09.540645 32517 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:06:09.540796 3795 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260926 02:06:09.866665 30884 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (556 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:06:09.866665 30748 heartbeater.cc:646] Failed to heartbeat to 127.29.89.189:33299 (557 consecutive failures): Network error: Failed to ping master at 127.29.89.189:33299: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:06:09.934435 31018 heartbeater.cc:646] Failed to heartbeat to 127.29.89.188:46477 (541 consecutive failures): Network error: Failed to ping master at 127.29.89.188:46477: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:06:13.084604 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:13.084748 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:18.087049 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:18.087164 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:23.089267 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:23.089382 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:28.091708 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:28.091840 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:28.565948 4073 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:06:28.566103 4073 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:06:28.566756 4073 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:06:28.567183 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:06:28.568130 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
W20260926 02:06:28.568224 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
I20260926 02:06:28.568287 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:06:28.568431 4073 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:06:33.094063 32500 master_runner.cc:187] Error getting master registration for 127.29.89.189:33299: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111),
I20260926 02:06:33.094172 32500 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 02:06:34.241425 4156 raft_consensus.cc:493] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 02:06:34.241593 4156 raft_consensus.cc:515] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "995d22fd6e1b4cdeb3b8aa4c61b56272" member_type: VOTER last_known_addr { host: "127.29.89.190" port: 45707 } } peers { permanent_uuid: "d95dbe14730b406b8f4ffced8abe38f2" member_type: VOTER last_known_addr { host: "127.29.89.189" port: 33299 } } peers { permanent_uuid: "b285db155484443594c9385155358d4d" member_type: VOTER last_known_addr { host: "127.29.89.188" port: 46477 } }
I20260926 02:06:34.242208 4156 leader_election.cc:290] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299), b285db155484443594c9385155358d4d (127.29.89.188:46477)
W20260926 02:06:34.242935 32518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 02:06:34.243768 32518 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d95dbe14730b406b8f4ffced8abe38f2 (127.29.89.189:33299): Network error: Client connection negotiation failed: client connection to 127.29.89.189:33299: connect: Connection refused (error 111)
W20260926 02:06:34.243844 32517 leader_election.cc:336] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b285db155484443594c9385155358d4d (127.29.89.188:46477): Network error: Client connection negotiation failed: client connection to 127.29.89.188:46477: connect: Connection refused (error 111)
I20260926 02:06:34.243884 32518 leader_election.cc:304] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995d22fd6e1b4cdeb3b8aa4c61b56272; no voters: b285db155484443594c9385155358d4d, d95dbe14730b406b8f4ffced8abe38f2
I20260926 02:06:34.244021 4156 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 995d22fd6e1b4cdeb3b8aa4c61b56272 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 02:06:35.340340 32522 maintenance_manager.cc:419] P 995d22fd6e1b4cdeb3b8aa4c61b56272: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 727 bytes on disk
I20260926 02:06:35.343304 32521 maintenance_manager.cc:643] P 995d22fd6e1b4cdeb3b8aa4c61b56272: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.002s sys 0.000s Metrics: {"dirs.queue_time_us":173,"dirs.run_cpu_time_us":84,"dirs.run_wall_time_us":85,"thread_start_us":189,"threads_started":2,"wal-append.queue_time_us":174}
W20260926 02:06:36.002908 31025 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
W20260926 02:06:36.003111 31025 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
W20260926 02:06:36.003186 31025 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
F20260926 02:06:36.013826 31040 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5f1c6f611ee8464b89f6dcd96dad2fc3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
*** Aborted at 1790388396 (unix time) try "date -d @1790388396" if you are using GNU date ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800007566) received by PID 30054 (TID 0x7f2373424700) from PID 30054; stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020826 31056 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020826 31056 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020848 31043 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020826 31056 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020848 31043 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020908 31053 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020826 31056 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020848 31043 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020908 31053 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020972 31035 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e297a6e8a69461b8c6d85132495df8c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f238aae8980 (unknown) at ??:0
F20260926 02:06:36.015936 31032 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_095de6750e1242cfb2364d77e2b4deb6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.017110 31038 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_67533d3acf714eba8041a0a5675dd2eb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018234 31041 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aba582f363ee48b38f7b72fe1507952f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.018342 31033 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fc856879f4024619ba0ceaa95ae58ad5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019383 31055 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019488 31034 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_28bfb85f5dab4406b28b8dc939814d7f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019518 31057 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.019533 31042 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_351548b11fca4255832809bbaf8f8a19 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020687 31036 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8917cd5381bb470e8ee76b412f3ec83a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020756 31039 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6cdde7bca9a8417091fa686ec9494cd4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020826 31056 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020848 31043 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020908 31053 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.020972 31035 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3e297a6e8a69461b8c6d85132495df8c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)F20260926 02:06:36.022033 31054 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 768) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.29.89.190:45707,127.29.89.189:33299,127.29.89.188:46477, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f2388657e87 gsignal at ??:0
@ 0x7f23886597f1 abort at ??:0
@ 0x7f23894c0d5d google::LogMessage::Fail() at ??:0
@ 0x7f23894c4b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f23894c075c google::LogMessage::Flush() at ??:0
@ 0x7f23894c1ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x56318baf2487 kudu::MasterStressTest::CreateTableThread() at ??:0
@ 0x56318bb04eb0 std::__invoke_impl<>() at ??:0
@ 0x56318bb02b9a std::__invoke<>() at ??:0
@ 0x56318bb0a10f _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTcl8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at ??:0
@ 0x56318bb09179 std::thread::_Invoker<>::operator()() at ??:0
@ 0x56318bb08196 std::thread::_State_impl<>::_M_run() at ??:0
@ 0x7f2388cdf6df (unknown) at ??:0
@ 0x7f238aadd6db start_thread at ??:0
@ 0x7f238873a61f clone at ??:0